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Preface 



"Higher Education In Georgia: Assess- 
ment, Evaluation, and Accreditation" is the 
theme of the third _research conference co- 
sponsored by the Office of jj»sti tut] qnal Re- 
search and Planning and the institute of 
Higher Education at the University of Geor- 
gia^ The conference was held January 15-16, 
1986 at the Center for Continuing Education 
and involved over 100 registered participants 
from over thirty-five different institutions of 
postsecondary education and ten educational 
agencies. 

The conference was conducted in co- 
operation jvith the Southern Association 
Colleges and Schools and addressed the as- 
sessment and evaluation implications of SACS 
new criteria for accreditation. The conference 
theme and [program topics were developed at 
a planning conference held at the Athens 
Hq I iday I inn on Novem ber 5 , 1 985 . Presen t at 
that planning conference were representatives 
of the various institutions and agencies who 
had expressed an interest in the research con- 
ference. The success of the third annual re- 
search conference thus is the result of the 
planning conference and the willingness of the 
planning participants to contribute to the 
later conference by preparing papers for pre- 
sentation. 

An expression of appreciation is in 
order for each program participant who pre- 
pared a paper for presentation at the research 
conference or who readily agreed to chair 



panel discussions. A special note of apprecia- 
tion is due Dr. James T. Rogers, Director of 
the Commission on Colleges, who discussed 
SACS new criteria for accreditation at the 
dinner and contributed substantially to the 
panel discussions the following day. His parti- 
cipation throughout the conference was espe- 
cially appreciated by those attending. 

Participant response to the research 
conference underscores the widespread inter- 
est in the research concerns and issues of 
postsecondary education in Georgia, and in 
conferring with colleagues with similar con- 
cerns and professional experience. One deci- 
sion made at the conference was to continue 
the annual conference, and it Is anticipated 
that fne fourth conference will be held in the 
fall of 1386. 

The editors of the proceedings are 
indebted to Donna Davis, institute of Higher 
Education, and Josephine Coile, Office of 
Institutional Research and Planning, for the 
competence and promptness with which they 
prepared camera-ready copy for the printers. 
The editors are also pleased with the coopera- 
tion of many others who have made the early 
publication of the proceedings possible. 



Cameron Fincher 
Larry G. Jones 
Joyce Placek 
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THE EMERGING ROLE OF ASSESSMENT AND EVALUATION 
IN POSTSECONDARY EDUCATION 

Cameron Fincher 
Regents Professor and Director 
Institute of Higher Education 



Some of us have been around enough 
to suspect that education is cyclic in its em- 
phases and interests. If we couldn't catch a 
challenging public issue in time to enjoy its 
moment before spotlights, we soon learned to 
hold onto bur lecture notes until the Issue 
came around again. It is not impossible that 
next time around, we can catch the same issue 
in its early returning stages and earn a bit of 
acclaim by saying again what we have said 
before. 

The current concern with assessment 
and evaluation may be such a topic. We are 
much concerned with educational outcomes 
as we were in the late 195% following the 
Equality Opportunity Study mandated by 
Congress in 1964 (Coleman, ei al., 1966). 
And we are much alarmed over the quality of 
public schooling as we were in 1957 following 
the launching of the Russian Sputnik. We are 
again aware thai education cannot be judged 
solely by the public resources allocated to 
schools and colleges, and certainly not by the 
internal processes of educational institutions 
and programs. We are once again in quest of 
hard, empirical, indisputable evidence that 
schools and colleges are instilling, "the^know- 
ledge, capacities, and skills" that students 
ought to acquire— and that society has a right 
to expect! 

In each case we are probably fright. But 
only because of the unbelievable complexity 
of secondary and postsecondary education in 
the United States, and because of our remark- 
able commitment to cultural pluralism In an 
increasingly diverse society. The quality of 
education is hot so mt»ch in evidence as the 
uneven diversity of educational institutions, 
programs, and personnel. The decline in aca- 
demic standards 5s not so much a problem as 
tne conflicting demands and confused 



expectations p[aced upon education in the 
last three years of secondary education and 
the first two years of postsecondary educa- 
tion. Even our demands for accountability 
have blown hot *nd cold, as we have confused 
Institutional impact with administration or 
fiscal respcnsibility, and instructional ac- 
countability with declines in student perfor- 
mance and/or academic standards. 

irrespective of our confused demands 

and expectations, we are nonetheless certain 
that the problems and issues of postsecondary 
education can be resolved through the uses 
and application of systematic and objective 
methods of measurement, assessment, and 
evaluation. We take for granted that such 
methods are available and that the pro* 
fessionaj or technical expertise of measure- 
ment assessment, and evaluation specialists 
can be Jjrought to bear on "a confidence 
cris|s" that apparently began with student 
protests^ test score decline, grade inflation, 
and other indications that all is not well with 
the nation's schools and colleges. For those 
of us with training and experience in testing 
and measurement, the renewed interest would 
be flattering if it were not blurred by a strong 
sense of deja vu. 

Measurement and Evaluation 

The push for measurement and evalua- 
tion was quite strong in the 1940s and the 
1950s L when i the measurement of individual 
differences was essential to the counseling and 
guidance of students in a democi^tic society. 
A strong emphasis was placed upon achieve- 
ment testing and the improvement of educa- 
tion through the construction and develop- 
ment of better education?! achievement tests. 
A course in testing was believed to be a 
necessary component cf teacher education. 
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in psychological testing a notion of general 
ability or general intelligence was countered 
by notions of special abilities that required 
separate tests. _ In addition to educational 
achievement and special abilities, there was an 
increasing concern with the emotional and 
social adjustments of students, as they adapt- 
ed to the demands of life, and with the mea- 
surement of attitudes, opinions, and interests, 
as such characteristics influenced personal 
decisions. These were the days in which thou- 
sands of school students took the Kaier 
Preference Record by punching holes with a 
metal pin in an answer pad and then counting 
the circles the pin had penetrated. 

These were also days in which a psy- 
chological concept of the whole individual 
had commendable influence Upon educational 
practices. Educational arid psychological 
characteristics were to be measured by sy- 
stematic and objective tests and inventories. 
Growth and development was to be evaluated 
by changes in test performance from various 
levels of formal education. A fairly common 
practice was to give some kind of readiness 
test at the beginning of the primary grades, 
followed by achievement tests in reading and 
arithmetic at the third or fourth grade levels. 
At the ninth grade level, it was common to 
test for specific areas of educational achieve- 
ment in such areas as science, mathematics, 
English, and history. Quite often these tests 
were used in courses called vocational gui- 
dance and students were given various kinds 
of assistance in their choice of career and 
educational programs. 

The purposes of evaluation, according 
to a classic textbook of the day (Remmers 
and Gage, 1955), were : 

1. To maintain standards, 

2. To select students, 

3 To motivate learning, 
4. To guide teaching, 



5. To appraise teachers, teaching methods, 
teaching books, curricula content, etc., 
and 

6. To furnish educational experience (p. 10). 



In brief, the purposes of testing were to select 
from among diverse individuals the one or 
more persons best suited for a particular job 
or educational program and conversely, to 
give to a single individual assistance in the 
many choices and decisions concerning career 
and education. 

The National Defense. Act of 1957 was 
indicative of the force., favoring selective 
admissions to educational institutions and 
programs. Top many students were regarded 
as deficient in science, mathematics, and 
foreign languages. As a result, a great empha- 
sis was placed upon recruiting students and 
admitting them selectively to enriched pro- 
grams in the three respective areas. As "the 
impending tidal wave 19 of the 1960s became a 
reality - f a greater emphasis was placed upon 
selective admissions. It was thought at that 
time that selective admissions would be the 
only way in which institutions of ppstseebn- 
dary education could cope with the increased 
numbers of applicants. 

At the same time testing and measure- 
ment were attacked viciously by a group of 
self appointed critics who published books 
under such salacious titles as, "the tyranny of 
testing," arid "the brain watchers./' Such an 
attack on testing came at a time when schools 
and colleges were _ increasing greatly the 
amount and kind of testing that they were 
doing for educational purposes. Under the 
confusions of the time, it is understandable 
why the uses of testing and measurement for 
the improvement of instruction were never 
quite realized. 

From Measurement To Assessment 

The changes in testing and measure- 
ment purposes may be seen by scanning E. F. 
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Lindquist's 1931 version of "Educational 
Measurement" and Robert L. Thorndike's 
1971 version of the same Book— both ver- 
sions, of course, Being published by the Amer- 
ican Council on Education. It is most interest- 
ing that the first chapter in the 1951 edition 
is entitled "The Function of Measurement in 
the Facilitation of Learning" and was written 
by Walter W. Cook of the University of 
Minnesota. The only comparable chapter in 
the 1971 edition is entitled "Measurement in 
Learning aridL Instruction," and was written 
by Robert G laser and Anthony J. Nitko. 
Thorndike uses the first chapter of the 1971 
edition to discuss iJie many changes thai have 
taken place in testing as the result of techno- 
logical development, the advent of large-scale 
testing programs, large-scale psychometric 
research projects, the development of instruc- 
tional systems, and other advances in testing 
technology. It js evident frbm Thbrndifce's 
chapter that testing in 1971 was greatly con- 
cerned with the formulation of behaviorial 
objectives, domain sampling, response sets 
and stylistic factors, and placement/classifica- 
tion applications as opposed to selection and 
prediction. 

In discussing the impact of testing upon 
education, Thorndike emphasized the inva- 
sion of privacy issue in the 1960s and the re- 
sulting fairness issue. The influence of deci- 
sion theory is much in evidence in the chapter 
on "Use of Measurement in Selection and 
Placement/ 9 written by John Hills, and in 
the closing chapter, "The Evaluation of Edu- 
cational Programs/' written by Alexander 
Astin and Robert J. Pahos. By the 1970s, 
educational evaluation was quite sensitive to 
the assessment of inputs, processes, and out- 
puts. The influence of Campbell and Stanley's 
"quasi-experimental" designs was also evi- 
dent and the principal purpose is readily iden- 
tified as information thai could guide deci- 
sions concerning the adoption or modification 
of an educational program. 

If we ask then what are the changing 
concepts of assessment, measurement, and 
evaluation as they evolved since WWII, we 
should appreciate some subtle distinctions in 



usage. Assessment— as the term is used— ap- 
parently means "to estimate, appraise, or 
size-up learner attributes or characteristics 
that cannot easily be measured. " Much of the 
time, bur assessment problems deal with such 
matters as personal experiences, learning 
strategies, and career aspirations. They con- 
tinue id deal, of course, with personal or 
social characteristics which can be judged but 
which do hot easily lend themselves to 
measurement 

Measurement apparently continues to 
mean to determine the amount, magnitude, 
extent, or degree of skills, competences, 
abilities, achievements, and accomplishments. 
Examples of these are the specialized skills or 
competences and the special abilities such as 
we find in technical education. It often in- 
cludes the basic academic skills such as read- 
ing writing, and mathematics— skills that 
ostensibly can still be measured. 

The term evaluation Joes not mean "to 
judge the merit or worth of changes in perfor- 
mance and or behavior/' as much as it does to 
determine the effectiveness of programs and 
projects that are designed to produce the 
changes. Evidently, we have evolved a usage 
in which we tend to evaluate programs and 
projects, but hoi id evaluate individuals or the 
change in their behavior or performance. To 
the contrary, we find [ahjhcreasihg tendency 
to use the word assessment for such purposes. 

Criteria That Must Be Met 

Irrespective of the^ subtle differences 
thai might exist among measurement, assess- 
ment, and evaluation there are criteria that all 
three methods must meet Among these crit- 
eria are those of systematic, objective, valid, 
and reliable observation and/or inference. In 
brief, assessment and evaluation in the 1986s 
must meet the traditional criteria that were 
specified for tests and measurements quite 
early in their development. 

Traditionally, we have Refined the ob- 
jectivity of tests and measurements as the ex- 
tent to which trained users can \gree on the 
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interpretation and use of results. We try to 
impose system and objectivity on assessment 
and evaluative efforts by specifying the con- 
ditions under which we will assess and evalu- 
ate. Validity is usually defined as the degree 
to which tests measure what they are intend- 
ed to measure. Another definition might be 
the precision and accuracy with which test 
scores designate performance or competence. 
In matters of test validity there has been a 
strong shift away from predictive validity and 
content validity, in favor of construct valid- 
ity. Beyond any doubt! the kind of Validity 
that assessment and evaluation methods 
should have in the 1980s is educational con- 
struct validity. 

Reliability— the stability or dependabil- 
ity of assessment or evaluation results— is 
still determined primarily in terms of internal 
consistency. There remain tod many difficult- 
ies in test-re test correlations in which an as- 
sessment instrument is used on different occa- 
sions. 

All assessment instruments must meet 
certain criteria pertaining to practical use. In 
short, the cost of assessment materials must 
not be prohibitive; there must be some prac- 
ticality concerning the administration, inter- 
pretation and reporting of assessment results; 
and there must be some concern with the 
distribution of interpretive information that 
will facilitate the decisions and choices that 
presumably will be made on the basis of as- 
sessment information. 

Two .le w criteria haVe been introduced 
since the 1960s arid are how criteria which all 
assessment instruments must meet These 
miry be identified simply as the criteria of (a) 
credibility and (b) fairness. In other words, 
there must be an absence of deception or 
cor ;ealment in the use and application of as- 
sessment methods. The reasons for the assess- 
ment must be obvious to those whose per- 
formance or competence are being assess- 
ed, the educational uses cf disguised, "trick- 
y" tests and exams have neVer been justified. 
At the moment, it is best to say hot only are 
they not justified, they may be illegal. 
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In much the same manner, there must 
be a removal of age, sex, and race biases that 
may enter into the content analysis and 
reporting, and use of assessment results. 
Although age, sex, and race continue to be 
variables that are related to assessment data 
and information, they must hot be interpre- 
ted as determinants and their influence on 
educational decisions, choices, and judgments 
must hot be dominant 

the changing emphases in testing and 
measurement thus represent a shift from a 
concern with measurement per se to the 
assessment of changes in behavior and perfor- 
mance that can be attributed to learning and 
development. Two other emphases that have 
been noticed are: (a) a shift from a concern 
with the evaluation of people to the evalua- 
tion of programs, and (b) a concern with the 
measurement of skills, abilities, or compe- 
tence as opposed to the measurement of apti- 
tudes. Each of these changing emphases in 
education should be welcomed. 

Why Assess? 

If we ask why we are so actively 
concerned with assessment, the reasons are 
the same that have been given in the past for 
our concern with testing and measuring. We 
assess in order to learn what students have 
learned; if they have learned what they have 
beer, taught; and how well they have learned. 
We also assess in order to make informed 
decisions and choices. Critics of testing often 
forget that teste, measurements, nnd/or 
assessments are the source of much of what 
we know about the individuality of students, 
their learning heeds and interests, their 
particular abilities and accomplishments, and 
their promise for continued growth and 
development Tests— standardized, commer- 
cially distributed, or teacher-made— are still 
the most acceptable means by which we learn 
what students have learned or achieved 'h six, 
nihe^ or eleven years of formal schooling. 
Assessment methods are now the best means 
by which we can Jearn if students have 
mastered, basic or fundamental skills af litera- 
cy; to what extent they have developed the 
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academic competencies they will need to con- 
tinue learning; and whether or not they have 
acquired the common core _or fund of know- 
ledge and information that many believe to be 
the foundation of education beyond the high 
school. 

But even more important, formal and 
informal assessment is the means by which 
students learn more about themselves. From 
teachermade tests, students can learn what 
teachers, school, and society expect them to 
learn— and whether or not they have barned 
what is expected of them. From assessment, 
students can learn more about their interests 
and aspirations, their persona! goals and ob- 
jectives, the fields of study in which they 
excel or experience difficulty, the level of 
their general educational development, and 
their potential for professional and personal 
development At least three generations of 
critics have underestimated the self-under- 
standing that ressujts from testing, measure- 
ment, an d/or assessment; it is very mu ch the 
baby in the bath water that they throw out. 

To Assess is to Compare 

The basis for all^ assessment is ^compari- 
son with some recognized standard of per- 
formance, the state expectations of society 
or its institutions, some previously established 
record or mark of accomplishment, the past 
achievement of individuals, or the group per- 
formance of some meaningfully defined crite- 
rion or norm group. In brief, all assessment 
results are relative to some identified standard 
or expectation, in education there are no 
absolutes and unless we know what we mean 
by standards, norms, and criteria, we will 
never be able to assess with any acceptable 
degree of credibility and fidelity. 

A confusion of the concepts: norms, 
standards, and criteria would seem to be at 
the bottom of our difficulties in defining 
educational or academic quality. Much too 
often the three terms are used interchange- 
ably with the generous assumption that 
listeners and readers will know what is meant 
by such terms. The evolving usage of the three 
terms should »ssen our confusion. Educa- 
tional standards, whatever they might be, 



should be recognized as^a stated or explicit 
expectation of performance or level of 
achievement, Norms L whatever they once 
were, are merely indjees or measures of typi- 
cal or representative behavior. Criteria are 
best regarded as the means by which we re- 
cognize that standards have been met; criteria 
are, , inother words, the evidence we introduce 
as a means of showing that academic stan- 
dards are in place and have been met 

it is most important to recognize that: 
(i j norms, standards, and criteria must be 
developed rather than discovered. This is 
merely another way of saying that they do 
not exist prior to their creation or their 
invention. eonver 'Sonai criteria such as grade 
point averages (GPA) have evolved through 
usage and experience, but at some time in the 
past the GPA was the creation and/or 
development of some imaginative educator. 

In much the same manner: (2) norms, 

standards, and criteria are seldom unitary. 
This merely means that there is no single best 
norm group, academic standard, or educa- 
tional criteria by which to judge student per- 
formance, program effectiveness, or institu- 
tional impact; (3) norms, standard, and crite- 
ria are abstractions. Each must be operational- 
ly defined for particu jar students, programs, 
and institutions. Our operational definitions, 
however, must not be too strict and there 
must be some degree of generalization to 
other situations and conditions. Otherwise, 
norms, standards, and criteria would not have 
die construct validity that we have mentioned 
previously. 

Finally, we should agree that: (4) 
norms, standards, and criteria for educational 
purposes should be educational in nature and 
consequence, it is necessary to say this be- 
cause of the over-emphasis that has often 
been place upon economic, social, and politi- 
cal outcomes in postsecondary education. 

To Assess is To interpret 

The value of all assessment methods 
must be demostrated in their use and applica- 
tion. It is possible to construct and develop 
an assessment method well; to assess student 
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performance, program effectiveness, or insti- 
tutional impact well; and then to invalidate 
the assessment process through the misinter- 
pretation of assessment results. This has often 
been the case with the measurement of cer- 
tain psychological characteristics such as in- 
telligence, and it continues to be the case with 
measures of academic ability such as the SAT. 
Indeed, the SAT is an excellent example of an 
assessment method for verbal and mathemati- 
cal abilities that is frequently undermined by 
faulty interpretation, use, or application. 

Effective assessment thus is dependent 
upon: |a) the purpose for which the assess- 
ment is being conducted, (b) the suitability of 
the assessment method or technique for the 
learners involved, (c) the training and ex- 
perience of those who are doing the assessing, 
and (d) the readiness of students, faculty, 
administrators, and policymakers to use the 
information gained from assessment 



With respect to assessment purposes it 
is necessary to say that all assessment 
should be explicit The primary pur- 
pose of assessment should be to esti- 
mate, appraise, or otherwise determine: 
the extent of student performance and 
competence in subjects and skills that 
they have been taught When assess- 
ment methods are used for purposes 
such as the evaluation of teaching 
effectiveness, curricula innovations, in- 
stitutional leadership, or the general 
condition of education, the purposes of 
such assessment should again be ex- 
plicit and the particular methods 
adopted and/or developed should be 
appropriate. 

The purposes of assessment should be 
determined by the educational objec- 
tives of programs and institutions. This 
is merely a restatement of the require- 
ment that assessment results be educa- 
tional in nature and consequence. 

The interpretation of assessment results 
is an acquired skill and should be so 



recognized. All participants in an assess- 
ment process and all potential users of 
assessment results should be given 
training and/or assistance in interpre- 
ting assessment results for their parti- 
cular purposes. Assessment results 
never speak for themselves, and like 
standardized test scores, no assessment 
result is self-interpreting. 

As in the establishment of an institu- 
tional testing program, a system of 
faculty evaluation, or the changing of 
degree or graduation requirements, am- 
ple consideration should be given to 
preparing the proper climate under 
which assessment and evaluation will be 
conducted. Just as in the classroom, 
where there should be no "pop quiz- 
zes", there should be no "pop assess- 
ments" of student performance, teach- 
ing effectiveness, or institutional im- 
pact. 

In brief, it should be obvious that the 
development and constructive use of suitable 
assessment methods in postsecondary educa- 
tion is a major challenge. We now have at our 
disposal numerous instruments, techniques, 
and procedures for assessing the various as- 
pects of learning and teaching, but we are still 
in need of more systematic, objective, valid, 
reliable, creditable, and fair assessment meth- 
ods. 

From Assessment to Evaluation 

The pressures for program and institu- 
tional evaluation stem from many of the same 
sources as those for assessment Nonetheless, 
there are some particular reasons for our great 
concern with assessment in the 1980s. The 
beginning of our concern may be dated from 
the mid-1960s when federal funding policies 
placed a premium on educational evaluation 
by writing requirements into the legislation of 
that t^ra. AH of us are familiar with the story 
of the Equal Educational Opportunity study 
that was mandated by Congress, hurriedly 
conducted in order to meet congressional 
datelines, then greatly publicized despite its 
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misinterpretation of findings. The EEO study 
and its subsequent publicity, however, was 
most effective in turning public attention to 
studies of educational outcomes and impact 
For colleges arid universities, program evalua- 
tion became a management imperative arid in 
the 1980s, program evaluation can be seen as 
a logical consequence of trends and events 
that took place earlier. 

In the 1960s, it was obvious a new re- 
search specialty was emerging in the form of 
evaluation research. In 1967 Michael Scriven 
made his distinction between formative arid 
summative evaluation, arid in 1969 Ralph 
Tyler edited a_ voJumefor the National 
Society for the Study of Education that gave 
strong emphasis to new concepts, procedures, 
and instruments for educational evaluation. 
The early 1970s produced many volumes of 
evaluation research in which explicit recogni- 
tion was given to: (1) the rieed for measurable 
outcomes, that were carefully specified in ad- 
vance, (2) feedback mechanisms that would 
permit adjustment or adaptations In academic 
programs, arid (3) the overall improvement of 
academic programs, teaching effectiveness, 
arid institutional viability. 



Why Evaluate? 

If we ask why bur great concern for 
evaluation, we receive much the same answer 
ffiat we did when we asked "Why Assess?". 
There is a need in the 1980s to reestablish 
the integrity and credibility of postsecondary 
education. A national concern for the quality 
for secondary schooling has shifted to post- 
secondary institutions arid three national 
reports may be mentioned as indicative of 
public awareness _ that all is not well in 
academe (NIE, 1984; ACC, 1985; arid NEH, 
1985). Despite the abundance of professional 
and technical literature dealing with program 
evaluation, the evaluation of academic 
programs arid the ensu ranee of academic 
quality pose many of the same problems as 
assessment. The primary purposes of evalua- 
tion are best identified as: (1 ) to describe, (2) 
to compare, and (3) to interpret. To describe 
institutional programs and performance with 



accuracy, we greatly need normative data that 
we often do riot have. Despite the great 
emphasis placed by the federal government on 
uniform reporting standards, we continue to 
lack systematic, objective forms of informa- 
tion about comparable institutions, programs, 
arid student populations. The comparisons of 
institutions and programs pose problems both 
similar to arid different from the comparisons 
of student performances we make in 
psychological arid educational measurement. 
Standards of institutional arid program per- 
formance are especially absent Arid the com- 
parability of much institutional and program 
data is more often suspect than demonstrable. 
Nonetheless, there is much we can do to im- 
prove the evaluation of programs and institu- 
tions in postsecondary education and im- 
provement should be bur proper concern. 

1. The evaluation of academic programs 
arid projects arid their impact upon 
institutional development should be an 
explicit responsibility of the institution 
itself. The use of outside consultants 
or evaJuators should be only brie 
component of the overall evaluation. 
No outside team or panel should be 
given the responsibility of evaluating 
institutional impact bi effectiveriess as 
such, but should confine its efforts to 
specific program effectiveness. 

2. All evaluation efforts should be more 
explicit about the need for evaluation 
as a continuing, ongoing activity in 
postsecondary institutions. There still 
is a tendency to think of evaluation as a 
one-stop service that can be provided 
within a span of months or a single 
academic year. Such a view usually 
produces frantic scrambling to assemble 
an evaluation team and to have them 
review academic programs and projects. 
In the case of funded projects, there is 
specious scurrying in order to submit 
an evaluation report before the expira- 
tion of a grant or its thirty-day grace 
period. 

3. Evaluation, irrespective of how quanti- 
tative or systematic the collection and 
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analysis of data_may be; eventually 
bojls down to a_matter of Human judg- 
ment It becomes a matter of what 
competent and creditable observers say 
about Jhe effectiveness of academic 
programs. The most significant state- 
ment that might be made, perhaps, is 
Sat program objectives are explicit and 
realistfc-and that program results are 
in keeping with those objectives. No 
amount of statistical data or quantita- 
tive analysis can alter this basic form of 
this interpretation. 

4. The effectiveness of evaluation is 
directly dependent upon the qualifica- 
tions and competences of those who 
are doing the evaluation. For this 
reason, we seriously contend that it is 
as Jmportant to know who the evalua- 
tors are, as jt is to know what it is they 
are evaluating. Because of this, serious 
consideration should be given as early 
as possible to the selection of evalua- 
tion teams or panels, and such teams or 
panels should be involved in the evalua- 
tion process as early as possible. 

5. The Indirect or deferred benefits of 
evahaation should be more explicitly 
recognized, tike planning, evaluation 
is a process and not a single event or 
result To facjjitate the indirect or de- 
ferred benefits of evaluation, those 
involved in the collection and analysis 
of evaluatjon data should know as soon 
as possible who members of an evalua- 
ting panel are going to be. This suggests 
that the purposes of evaluation are 
always, to some extent, specific and/or 
particular. 

6. The question of institutional impact is 
not one that should be considered in 
most evaluation efforts. Academic 
programs and projects, when con- 
sidered in gelation to their explicit 
purposes, may be quite successful 
without giving visible evidence of in- 
stitutional impact at the time of evalua- 
tion. Institutional impact is thus an 
outcome that should be evaluated 



separately in an entirely different 
manner. 

7. the criteria of evaluation should^ Be 
more explicit about the Important 
functions of interviewing and observa- 
tion during onsite visits. The impor- 
tance of these skills ^ emphjoize the 
necessity of identifying early the 
members of evaluation teams or pane|s. 
Perhaps the evaluators' vito should be 
presented as part of proposals for 
funded projects so that funding agen- 
cies can see exactly who will serve as 
evaluators on funded projects. 

8. Evaluation criteria should also be more 
explicit about the naturejrf the evalua- 
tion report: to whom the report wij| 
be made; and the conditions under 
which the report will be submitted. It 
is often advisable that part of the eva- 
luation report be in writing— and part 
of it be unwritten. In other wordSj 
there may be unpleasant news that 
should be communicated in confidence 
to presidents, deans, or department 
heads and hot made a matter of record. 
The point to be made is that evaluation 
requires a relationship of trust and 
confidence in which evaluators can say 
uncomplimentary as well as compli- 
mentary things about programs and 
projects, and about the performance of 
professional staff. 

9. The experience and expertise of evalua- 
tion teams or panels are, of course, 
paramount The professional experi- 
ences of members should be diverse, 
but their expertise should be comple- 
mentary, in other words, evaluation 
probably is not effective when all 
members of evaluation teams or panels 
think as one. 

10. A better understanding of the role and 
responsibilities of evaluators should be 
worked out if the evaluators are to 
serve as an intermediary between 
funding or governing agencies and a 
funded institution or program, this 
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should be explicitly stated from the be- 
ginning. Evaluation is riot a service in 
which there should be conflicts of alle- 
giance. 

11. Lastly, the evaluation itself must be 
evaluated. A part of the general evalua- 
tion of academic programs and projects 
should always include some considera- 
tion and statement of the effectiveness 
of the evaluation itself and a statement 
of the institutions! satisfaction with the 
performance of outside consultants 
and/or evaluators. in much the same 
manner! funding agencies and governing 
boards should clearly communicate 
their satisfaction, or lack thereof, to 
the evaluator. That communication 
should be explicit enough to inform the 
evaluators how well they have served 
the evaluated institution, program, or 
project 



From Evaluation To Measurement 

The preceeding discussion should 
suggest strongly that the effectiveness of insti- 
tutional and program evaluation is very much 
dependent upon the development of effective 
assessment methods, techniques, and 
instruments. The methods developed may be 
formal or informal, nationally developed or 
institutionally-specific, and qualitative or 
quantitative, (i.e., they may deal with cate- 
gorical data that are intensive/qualitative as 
opposed to continuous data that are exten- 
sive/quantitative). A diversity of approaches 
are permtssable in die evaluation of academic 
programs, but all approacnes must be 
supported by evidence that they: (1) have 
been well-thought-out; (2) encourage inter- 
personal agreement among knowledgeable 
observers and participants; (3) reflect educa- 
tional purposes arid concerns that are relevant 
to learning and teaching; (4) result in con- 
sistent or dependable findings and inferences; 
(5) meet the logical expectations of those af- 
fected; and (6) violate no one's sense of fair- 
ness. In other words, and obviously, methods 
that are systematic, objective, valid, reliable! 
credible, and fair. 



Also important in the application \ of 
assessment methods in program evaluation 
are: (1j the development of a rationale that 
clearly states the purposes of evaluation, (2) 
the careful preparation of a climate or en- 
vironment in whjch the evaluation of 
programs and personnel can take place, (3) a 
well-planned fol low-through [ that will ensure 
the continued effectiveness of evaluative 
efforts. In particular, the purposes of assess- 
ment and evaluation should 1^ be carefully 
delineated from program objectives and in- 
stitutional goals. Unless circumstances fully 
warrant other evaluation purposes, program 
evaluation should always be conducted with 
regard to the explicit purpose of the acade- 
mic program or service. Another way of say- 
ing this is to reject notions of global, totalistrc 
evaluation that tell us nothing about the ef- 
fectivenss with which observed outcomes 
match expected outcomes of funded pro- 
grams or projects. Programs objectives should 
be approved at the time that programs are 
authorized and/or funded; programs should 
then be evaluated in terms of how well they 
accomplish the purposes for which they 
were established. 

To assess and evaluate student perfor- 
mance, academic programs and services, and 
institutional effectiveness in the manner 
considered here, many of us can see a re- 
turning emphasis on basic concepts and prin- 
ciples of measurement The Southern Region- 
al Education Board (SREB), in its 1981 re- 
port on "The Need for Quality" was the first 
educational agency to identify a national need 
to improve the quality of education at all 
leve|s. It was the first public commission to 
state unequivocally that should imjjlement 
minimum standards across the board and then 
surpass them, in its report, SREB further 
stated that current college admission stan- 
dards were no demand for quality. In 1983 
and 1984 a dozen prestigious national com- 
missions, task forces, or panels were saying 
the same things. 

In 1983 the chairman of SREB's 
Commission on Educational Quality, Gover- 
nor Lamar Alexander of Tennessee, stated his 
conviction that the South should lead the 
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nation in "measuring education progress. 19 
Explicit in the expectations of public leaders 
was the use of educational achievement tests 
that would permit the comparison of student 
achievement in southern states with the per- 
formance of students nationally or in other 
states. In short, southern governors how 
wanted standardized test results that would 
demonstrate the comparative progress of 
education in their respective states. 

To demonstrate in measureable terms 
the progress of education in southern states, 
senior exit or proficiency exams would be 
heeded. More attention should be given to 
entry-level tests, and more states should 
follow the lead of Georgia and Florida in 
establishing statewide or systemwide testing 
programs at the conclusion of the sophomore 
year in college. It is all reminiscent of the 
testing programs developed at Georgia State 
in the early 1960s. 

At that time Georgia State had the 
most comprehensive testinj^prbgram of any 
institution in the nation. The objectives of 
thai program were to provide assistance in: 
(1) selecting students with qualifications suit- 
able for the various degree programs offered 
by the institution, (2) placing students with 
comparable ability in courses adapted to their 
level of academic achievement, (3) advising 
students about degree requirements, college 
regulations, and curriculum opportunities, (4) 
counseling students as to vocational arid 
educational objectives, remediation of scho- 
lastic difficulties, and optimum use of their 
abilities and talents, (5) assessing the acade- 
mic progress made by students at various 
stages of educational advancement, and (6) 
evaluating the effectiveness of academic 
programs offered and the utility of curricu- 
lum requirements. [Note: all terms above 
were the terms used to describe the program 
at that time.] 

All entering freshman were required to 
take the: 



1. College Board Scholastic Apti- 
tude Test (SAT) 
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2. Otis Quick-Scoring test of Mental 
Ability 

3. Cooperative General Achievement 
Tests (in socfal studies, natural 
science, and mathematics) 

4. New Cooperative English Test: 
English Expression 

5. Nelson-Denny Reading Test 



At the "rising junior" level all students 
were required to take the: 

1. Cooperative General Culture Tests 
(in social studies, literature, 
science, mathematics, and fine 
arts) 

2. Cooperative English Expression 
Test 

Graduating seniors in the College of 
Arts and Sciences took (at institutional 
expense) the: 

1 . Graduate _ Record Examination 
General Jest {for verbal and 
mathematical abilities) 

2. Graduate. Record Examination 
Subject Test (in areas appropriate 
to the student's particular field of 
study) 

It should be needless to add that GSU's 
testing program was expensive, time-consum- 
ing, and burdensome. It was also highly 
effective— and very useful in matters of 
program assessment and/or evaluation and in- 
stitutional accreditation and advancement! It 
did have a bearing on academic programs and 
the education of students. 
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ASSESSING STUDENT ACHIEVEMENT 



J oe Marks 
Southern Regional Education Board 
Atlanta, Georgia 



Over most of the history of American 
higher education, attending college was a 
irivilege for those individuals who were be- 
lieved to be destined to leadership roles in 
society—mainly the brightest among the af- 
fluent Students progressed under the scrutiny 
of the faculty who monitored the course of 
study and were called upon to attest to the 
students' achievement, as signified by the 
award of grades and, uitimateiy, a degree. The 
assessment of student achievement was, in 
almost all easel, left to the faculty. The higher 
education community judged institutions' 
quality by their selectivity. The most selective 
institutions enrolled the most promising stu- 
dents, and it was assumed that those who 
graduated were necessarily the best educated; 
hence^ from the best institutions. 

The emergence of access as one of the 
primary goals of American higher education 
and the growing reliance of the American 
economy on a college educated workforce has 
made selectivity a less relevant indicator of 
quality. More and more* questions are being 
raised about the performance levels of todays' 
college students and graduates, in particular, 
our states' elected decision makers are seeking 
ways to formalize the assessment of student 
achievement in the hope of providing an indi- 
cator of quality more relevant to the context 
of higher education today. Possibly in re- 
sponse to this new questioning, the South 
regional accrediting association has taken the 
significant step of including, in its revised 
standards, criteria that require higher educa- 
tion institutions to document the achieve- 
ment of their institutional goals— efforts 
certainly to entail increased assessment of 
student achievement 



A Model for Analyzing Assessment Practices 

There are three major areas of student 
achievement chat are subject to assessment: 
intellectual development, career development, 
and personal development (Figure 1). From 
an educational standpoint, the primary area 
of achievement is intellectual development 
What basic skills have students mastered? 
What levels of academic attainment have stu- 
dents reached in general and in specialized 
knowledge? What special aptitudes have stu- 
dents developed? 

The second and third areas in which 
student achievement is assessed are career and 
personal development. What levels of career 
aptitudes and awareness have students acquir- 
ed? How many years of education do students 
finally complete? What are students' voca- 
tional achievements, such as level of responsi- 
bility, income, awards and special recogni- 
tion? The personal development area covers 
self-concept, attitudes, beliefs, and value 
systems. How prepared for life and how 
suited for citizenship have students become? 

in addition to there being different 
areas of student achievement which are sub- 
ject to assessment, there is the question of 
which stage, or stages, of student develop- 
ment are subject to assessment There are as- 
sessments which measure aptitudes and 
achievements of first-time students, those 
which mark progress made by continuing 
students during the college-going period, and 
those which measure graduating students' 
achievement. Assessment results might be 
comparable among states and institutions or 
they may be one-of-a-kind, with no 
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Figure 1 

AREAS, LEVELS, AND INSTRUMENTS OF STUDENT ASSESSMENT 

SREB STATES 



Areas of Development 

Levels of 

Development INTELLECTUAL CAREER PERSONAL 



GRADUATING GRE 
STUDENTS LSAT 

MCAT 

NTE 



CIRP CIRP 
SOIS SOIS 

ESS 



CONTINUING COMP 
STUDENTS NTE 
(Soph/Jun) PEP 

PPST 



Aptitude & CIRP 
Career Guidance 
Tests ESS 

SOIS 



FIRST-TIME ACT 
STUDENTS APP 

CLEP 
SAT 



Aptitude & CIRP 
Career Guidance ESS 
Tests SOIS 

SAT & ACT 
Profiles 



KEY: Graduate Record Exams (GRE); Law School Aptitude Test (LSAT); Cooperative 

Institutional Research Program (CIRP); Student Outcomes Information Services 
(SOIS); Medical College Admission Test (MCAT); Evaluation Survey Service (ESS); 
National Teacher Examination (NTE); College Outcomes Measurement Program 
(COMP); Proficiencies Examination Program (PEP); Pre-Professional Skills Test 
(PPST); American College Test (ACT); Advanced Placement Program (APP j; 
College Level Examination Program (CLEP); Scholastic Aptitude Test (SAT). 

SOURCE: Southern Regional Education Board. Measuring Educational Pro g r e ss in the Sou th : 
Student Achievement (Atlanta, Georgia: SREB, 1984). 
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comparability to others. In addition, assess- 
ment might be required or conducted on a 
systemwide basis or employed by institutions 
on a discretionary basis. 

Assessment Practices in the SREB States 

The most widely practiced assessments 
are of first-time college students 1 intellectual 
development Nearly all Southern colleges and 
universities require first-time students to 
submit Scholastic Aptitude Test (SAT) or 
American College Test (ACT) scores. These 
do not measure what students learn in college, 
out indicate academic aptitudes and prior 
achievement. Scores ate used to evaluate pro- 
spective students for admission and to aid in 
academic placement While perhaps less of a 
factor in admissions today r these tests are in- 
creasingly looked at to reveal the educational 
preparation of college-bound students. Some 
SREB states use these "entrance exam" re- 
sults on a statewide basis. For example, SAT 
freshman norms are distributed throughout 
Se U in ^ersity System of Georgia and, in a 
"high school feedback" program, each high 
school receives a status report, including SAT 
and early course performance information, 
about its former students. 

Many states also require the Test of 
Standard Written English (TWSE), and the 
California Achievement Test's (CAT) math 
section for students who score below a certain 
level on the ACT or SAT- Another widely 
practiced assessment is evaluation of first-time 
students to determine if they should be 
awarded college credit for knowledge already 
attained (e.g., College Level Examination Pro- 
gram-CLEP). The Test of English as a 
Foreign Language (TOEFL) is widely used to 
evaluate foreign students for admission and 
academic placement. 

Beginning mid-1984, Florida imple- 
mented common placement tests and testing 
procedures to assess the basic computation 
and communication skills of all students en- 
tering college. Cut-off scores determine which 
students require remediation. The University 
System of Georgia developed,many years ago, 
a basic skills testing program for this purpose. 



In addition to these types of requirements, 
several states are looking into or are requiring 
first-time students to have completed a mini- 
mum number of prescribed secondary units 
for college entry. 



More and more assessment of student 
progress in college is being conducted for en- 
trance to teacher education, nursing, and 
other specialized programs. Most employ 
comparable testing instruments, such I as por- 
tions of the National Teacher Examinations 
(NTE), the Col lege Outcomes Measurement 
Program (COMP), the Proficiencies Examina- 
tion Program (PEP), the Pre-Professional 
Skills Test (PPST), and the California 
Achievement Tests (CAT). Five of the South- 
ern states— Maryland, Mississippi, North Caro- 
lina, Tennessee, and Texas—now require 
entrance exams other than SAT or ACT for 
<tu dents pursuing a teacher education degree, 
Alabama, Arkansas, Georgia, Louisiana, and 
Virginia use SAT or ACT scores with grade 
point average to determine eligibility. Non- 
comparable assessments are used as well. In 
Kentucky, the State Board of Education, 
which regulates teacher certification, requires 
basic skills tests for admission to teacher edu- 
cation programs. Individual institutions select 
their own instruments and determine mini- 
mum scores. 

A widely noted type of student 
achievement assessment is the "rising junior" 
test. Florida and Georgia have received na- 
tional attention for being among the first 
states to develop such assessments. The 
Florida "rising junior", test— College Level 
Academic Skills Test (CLAST)— assesses con- 
tinuing students' communication and compu- 
tation skills. The jest is based on faculty 
consensus about the skills appropriate for all 
students moving to the junior level. Over 
time, the score required to qualify students to 
be eligible to receive an associate degree or to 
be given upper-division standing is being 
raised. The requirement applies to transfer 
students as well, and this academic year stu- 
dents enrolled in Florida's independent insti- 
tutions must participate if they receive state 
financial aid. 
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Georgia's Regents' Testing Program, 
a!so considered a "rising Junior" test; ;_h in- 
tended to assure that all graduating students 
have certain minimum skills in reading and 
writing. Reports on student performance and 
comparisons between institutions are distri- 
buted for planning purposes. Remedial 
courses are available— or required for students 
who have 75 hours of degree credit and have 
hot passed. 

Rising junior tests are essentially mini- 
mum competency examinations intended to 
ensure a certain level of achievement for all 
college students. Like the high school mini- 
mum competency graduation tests, they do 
not measure "high quality." !n fact, one wide- 
spread concern about tile use of minimum 
competency tests is the worry that minimum 
competency may become maximum expecta- 
tion. 

Tennessee is the only Southern state 
with a statewide program to assess all graduat- 
ing students. They administer the College 
Outcomes Measurement Program i (COMPj to a 
sample of four-year college graduates. 



At least half of the SREB states assess 
graduates of specialized programs. Arkansas, 
Louisiana, Mississippi, North Carolina, South 
Carolina, Tennessee, and Virginia use the 
National Teachers Examinations (NTE) for 
graduates seeking teacher certification. Virtu- 
ally all the other states use teacher certifica- 
tion tests of some kind. While the passing 
score varies from state to state, in general it is 
set at a very low level— far below the national 
norms (if any) on the teste; the idea is, again, 
to assure minimum competency. 

Other assessments of graduating stu- 
dents include the widespread use of the 
Graduate Record Exam (GRE) or the Miller 
Analogies Test (MAT) for students seeking 
entrance to graduate school. Some under- 
graduate departments are experimenting with 
the GRE as a requirement for all their gradu- 
ating students, with both locally and external- 
ly developed senior exar is, with peer evalua- 
tions, or with COMP. 



The growing importance of student as- 
sessment is illustrated by the fact that some 
states— Alabama, Florida, Georgia (and 
Ten nessee in 1 ^86j— are using certification 
test passing rates for graduates of teacher 
education programs to make decisions about 
continuing state approval of the programs. 
State higher education agencies have also used 
nurse licensing examinations in making deci- 
sions about whether to continue state appro- 
val for nursing programs. With the growing 
interest in student performance, more atten- 
tion to the results of these important exam- 
nations is expected. 



Concern for the areas of career and 
personal development results almost exclu- 
sively from institutional initiative. SAT and 
ACT profiles— which include student back- 
ground information as weP as test scores— are 
widely used. Standardized aptitude and career 
guidance tests are used, sometimes as part of 
the admissions process. Counseling and gui* 
dance centers have a broad variety of tests 
available. Follow-up studies are also conduct- 
ed widely to assess the career development of 
graduates; though few of these follow-ups are 
system-wide or employ comparable question- 
naires. 

Conclusions 

The assessment of student achievement 
in higher education is less extensive than in 
elementary and secondary education. The 
suggestion of putting a greater emphasis on 
assessment has even raised an outcry in some 
higher education circles. Yet, even now., high- 
er education assessments of student achieve- 
ment are the basis of important decisions 
affecting institutions, students, and society at 
large. Decisions such as who will attend which 
institutions of higher learning, who will be 
allowed to prepare for specific professions, 
who will be certified jn the professions— and 
in some cases— who will be allowed to receive 
college degrees, and which institutions will 
receive extra funding or state approval for 
certain programs. Today there is interest in a 
new form of accountability for higher educa- 
tion—accountability on the basis of the 
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demonstrated achievement of students, not 
jiisu on financial criteria; and quality judg- 
ments on the basis of student academic suc- 
cess, not just on the basis of selectivity; 

Most formal assessments of student 
achievement at die collegiate |eve[ still occur 
at the college admissions stage. Trae college- 
level assessment of student serves special 
categories of students, such as those seeking 
credit for knowledge already acquired 
(CLEF), or entrance into specific programs 



(NTE or 6RE) ; serves a "jpte keeping" func- 
tion aimed at certifying minimum academic 
accomplishment, e.g., certficaticm tests for 
graduates seeking to teach, or "rising junior" 
tests for college sophomores; serves as a basis 
f or the evaluation of specific programs, e.g., 
teacher education or nursing, or for decisions 
about state approval or authorization; ard, at 
a more or less experimental fevei, to monitor 
improvemer»t^ rn student performance and 
educational pi ogress. 
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__Tb ol7, 35 E. R. Redmon 
Souther i Region?! Education Board 
Atlanta, Georgia 



It is difficult to be pessimistic about 
the ultimate outcomes of education at any 
level in the U n ited States after one reviews 
the recommendations and rhetoric of some 
two dozen national _ reports on current and 
future plans to improve the opportunities to 
learn and the commitments to teach in our 
colleges and schools. And yet, the disarray of 
the current moment is evident enough to pur- 
suade most parents, students, and teachers 
not to view education as their sole soi rce of 
personal and career development indeed, if 
th e na ti on 9 s in vestment in education we re 
situated in stocks and bonds, there could be 
no doubt that the bears would be in control 
and that the general climate of education 
would represent the near Jjottor.i in a de- 
pression that began not too many years ago. 



The Definition of Assessment 

Assessment appears to Be not only a 
useful term in the education dictionary but 
alsr* opt only way out of an otherwise un- 
measurable decline^ in the benefits of this 
most treasured^ American entitlement. Re- 
cently, the term has gained credibility, bigh- 
[ighted by an October meeting of educators 
responsible for assessment in Columbia, South 
Carolina, sponsored by the American Council 
on i Education. Whatever gain may be realized 
from th is nati on 's climb tH2 t of a depressive 
educational era appears to be linked closely 
wi^ Ae effective u« of assessment as a tool 
to measure our future progress. 

Traditfonally, assessment has been used 
to make reference to any number of mea- 
surab[e scdes of progress. The measurable 
progress might be i related to achievement or 
to some other structural improvement or 
change in institutions or in state responsibility 
for education. 



Two examples come to mind as the 
most widely used assessment activities of 
current interest to institutional decision 
makers. First, the increasing use of tests of 
academic performance is reaching an all-time 
high on coitege carr<puse^ These tests of aca- 
demic ability are used at the admissions level 
all the way through "exit" levels from the 
sophomore or senior year of college. Perhaps 
the tradition of classroom evaluation lies at 
the heart of this increasing u$£ of perfor- 
mance measurement Certaphly the concept 
of testing students on their achievement and 
ability does not require extensive explana- 
tion or defense in faculty senates or other 
campus decision groups. Our abiding faith in 
objective (and now essay) testing contributes 
to this long-standing, traditional method of 
assessment. 

The use of such tests requ ires a number 
of important decisions about the cho'ce of an 
institutionally developed instrument or the 
application (and perhaps modification) of a 
generally available commercial product Re- 
gardless of the choice, conscientious admini- 
strators and faculty have learned what it takes 
for such testing to produce useful outcomes. 
A brief review of such experience indicates a 
number of requirements for any assessment 
by way of objective or essay testing of aca- 
demic performance: 



1. Clear purposes regarding the overall use 
of tests for particular application as 
well as the choice of a particular in- 
strument to measure defined achieve- 
ment or ability. 

2. Clear question specifications allowing 
the matching of measures to expected 
outcomes. 
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3. Comprehensiveness of the instrumen * 
to adequately jneasu ire ability or a- 
chievement in the areas of concern. 

4. Explicit connections to classroom set- 
tings, instiuctir lal method and course 
structure. 

5. An informed decSsson regarding the 
choice of essay and/or objective items 
as the forms of evaluation. 

6. Clear direct ; ons jjtj dents will under- 
stand without lengthy explanation. 

7. A well-articulated policy relating the 
outcome of test scores to overall 
evaluation by grades. 

8. Opportunities to evaluate tfte test 
according to the difficulty of items on 
the instrument as well as the validity of 
the 5n. trument for the purpose defined. 

Although many of these requirements are 
taken into consideration at most campuses, 
assessment is not always consistent and the 
use of such requirements becomes burden- 
some in many settings of evaluation. 

Additional concern also should be fo- 
cused on at least three areas of validity of 
such instruments: 

Construct validity (making sure the 
idea of using a test fits the circum- 
stances for evaluating academic out- 
comes) 

Content validity (making sure that the 
evaluation instrument chosen or cre- 
ated fits the specific instructional and 
curriculum setting within which stu- 
dents are to be evaluated) 

Predictive validity (making sure that 
the test will serve the purpose of certi- 
fying preparation for the next step of 
education in a specific setting) 

A second typical area of assessment 
that has evolved to Judge the effectiveness of 



college may be found in many assessments of 
student satisfaction with college. Although 
the evaluation typically is administered in the 
form of a survey (with an infinite variety of 
such instruments used at hundreds of colleges 
across the nation)! the primary purpose of 
such an effort is ic ascertain the level of safsr 
faction students have wit!: their under- 
graduate experience. The quality of such 
assessment generally depends upon the so- 
phistication of the survey research design and 
the expected use of the outcomes of the sur- 
vey. Bea! and Noei (1979) found 5ri a 
follow-up study of student satiWcion with 
their undergraduate experience :igniffcant 
factors relating to the retention of students, 
their findings suggest the value o' such as- 
sessment arid its impact on institutions with 
significant retention problems. fpe<;ifical!y, 
this example of assessment yielded five major 
negative factors which students find to be un- 
satisfactory. 



1. 


Inadequate academic advising 


2. 


inadequate curricular offerings 


3. 


Conflict between class schedule and job 


4. 


inadequate financial aid 


5. 


inadequate counselling support systems 


6. 


inadequate extracurricular offerings 




Conversely, the study also found cer- 



tain factors to be positively related to stu- 
dents' satisfaction with their undergraduate 
experience. 



1. 


Caring attitude of faculty and staff 


2. 


High quality of teaching 


3. 


Adequate financial aid 


4. 


Student involvement in campus life 


5. 


High quality of advising 




Regardless of whether institutions use 



these two examples of assessment or other 



Red m on 



21 



traditional forms of measuring student satis- 
faction and ability, there appear:; to be great 
comfort taken in simply doing the testing or 
administering the survey. The real value (and 
difficulty) of such assessment is in the ability 
of a college or state system of higher educa- 
tion to effectively incorporate those out" 
comes into the provision of the best possible 
educational services to student?. 

While many anthors assume that ths 
outcomes of education are related directly 
to measurable scales on tests or other evalua- 
tion insmjments, this paper will suggest the 
larger definition of an educational outcome as 
a goal related to the overall purpose of educa- 
tion in any specific setting. For state and in- 
stitutional; poiic /makers, the definition of an 
education outcome as a measurable state- 
ment of a goai yet to be accomplished 
provides two important foundations for 
research and policymaking: 

1. The outcome of a particular educa- 
tional activity may be perceived as a 
clear statement of what ought to be; 

2. The definition of outcomes as specific, 
measurable goals may allow for any 
number of evaluation or testing techni- 
ques to be used to determine if progress 
is underway or if the outcome has been 
achieved, 

Such a definition allows outcomes to 
be used more directly to formulate educa- 
tional policy and to make use of more than 
one specific indicator of progress or attain- 
ment Further, this view allows educational 
outcomes to be studied with respect to their 
costs and benefits to individuals arid institu- 
tions and to the society that supports them. 
As Howard Bowen (1977) has cautioned, 
educators can make a significant mistake "by 
assuming that improved outcomes are desir- 
able regardless of cost ... In assessing the 
efficiency of higher education, , . . one must 
consider both cost and result" (p. 21 j 
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Planning For Desired Outcomes 

Descriptive research on higher educa- 
tion yields ? wealth of data on the effectsof 
education. Such research, Based on time- 
proven evaluations, wiH continue to give the 
public and its policymaking representatives 
handholds on educational improvement ef- 
forts. But the effects of education often have 
little to do with the desired outcomes em- 
bodied in state plans and institotjonS mis- 
sions. _ As a comprehensive example of the 
kind of outcomes that education should yield, 
the Southern Regional Education Board's 
recommendations through its Commission on 
Educational Quality are worth careful review; 

In 1985, the SREB Commission pub- 
lished 50 specific goals in three important 
areas of educational reform: 

1. Providing access to quality under- 
graduate education 

2. Improving teacher education in colleges 
arid schools 

3. Setting academic standards for sec- 
ondary vocational education 

These three areas of educational reform 
riot only organize useful groups of educa- 
tional outcomes but appear (on the basis of 
recent responses) to provide direction for 
educational activities in the most critical (de- 
pressed) arenas of educational quality. 

Undergraduate Education 

For several decades, American higher 
education has found itself in the spotlight of 
educational reform, Since the Truman €om- 
mission on Higher Education arid the Harvard 
"redbodfc", countless institutional, state, and 
federal reports have called for moderate Jo 
dramatic tinkering with what had become 
known as undergraduate general education; 
The current spate of reports offered to 
various audiences to confirm that rigorous 
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self study is underway continues that tradi- 
tion of appraisal from within and without. 
The SREB goals call for specific actions, or if 
you will, outcomes, in key problem areas. 

Setting standards for college-level aca- 
demic performance. (Including specific 
recommendations on entry-level stan- 
dards and indicators of satisfactory 
progress throughout the undergraduate 
course of study) 

Making college remedial programs ac- 
cessible to students, (including recom- 
mendations regarding both the geo- 
graphical and financial accessibility of 
college preparatory study) 

Preparing students for college. (In- 
cluding strongly worded recommenda- 
tions for increased cooperation 
between high schools and colleges, and 
the development of explicit measure- 
ments of adequate preparation for 
college-level work) 

Strengthening the college curriculum. 
(Specifying goals for content reform in 
undergraduate curriculum and under- 
lining the importance of faculty co- 
operation in reform activities) 

Preparing faculty for teaching under- 
graduate studies. (Highlighting the im- 
portance of faculty attitudes and skills 
related to teaching undergraduates 
during the first two years of college 
work) 



Teacher Education 

The Commission Report on improving 
Teacher Education included a large number of 
goals related to four major agendas for higher 
education and the schools: 

Recruitment of talented students, (in- 
cluding concerns for adequate academic 
preparation of students and appropriate 
incentives to attract students into edu- 
cation for the teaching profession) 



The content and structure of pro- 
fessional education. (Twelve of the 
goals in this report focused on tfiejiro- 
gram structure, the content of study 
and the practical experience that form 
the substance of professional education 
for teachers) 

Institutional support (including goals 
on making teacher education improve- 
ment a priority agenda Item for presi- 
dents and chancellors, increasing insti- 
tutional support for faculty in colleges 
of education, and advocating increased 
contact between professional teacher 
educators and public school administra- 
tion and teaching) 

State support, (including concern for 
public support for teaching, state ex- 
ecutive support and increased legislative 
funding for teacher education pro- 
grams) 



Vocational Education 

The Commission's Report on Impro- 
ving Education included goals in two major 
areas: 

Maintaining quality and raising acade- 
mic standards. (The goals included 
concern for restructuring the linkage 
between vocational and academic edu- 
cation, course design, basic skills as- 
sessment and remediation, and the 
important relationship between prac- 
tical experience and academic learning) 

Experimenting with new models. (The 
recommendations suggested the forma- 
tion of state task forces to encourage 
pilot projects to improve secondary 
vocational education, increasing atten- 
tion to evaluation, joint enrollment 
opportunities, upgrading the use of 
tech no! ogy in vocational educati on , 
and exploring new structures of co- 
operative education) 
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Planning and Con trolling Outcomes 

As in any organizational system, there 
is no reason to spend scarce resources on out- 
comes that do not help fulfill the goals of the 
organization. Underneath the interest in 
higher education institutions for calm, neutral 
research, colleges and states want something 
good to happen to the students who enroll. 
Much of that good can be found in the above 
statements of goals with which institutions 
will become committed in various ways over 
the next decade. This paper will conclude 
with a call for concern not so much for the 
inevitable choices of desired outcomes but for 
increasing commitment to plan for and 
control those outcomes in order that they 
will, in fact, occur. 

In the October 1985 ACE meeting on 
assessment in Columbia, South Carolina, 
Professor John Harris of David Lipscomb 
College presented an excellent overview of 
assessment. He introduced his presentation 
through a letter to Dr. Dubious Scholarius, 
Dean of the Faculty and Vice President for 
Academic Affairs at Everyone University, 
Oxbridge, U.S.A. In his contrived letter, he 
recognized Dr. Scholarius' skepticism about 
"the whole outcomes business", but went on 
to assure the Dean of how important his 
interest in this area can be to the mission of 
Everyone University. 

in John Harris' closing paragraphs, we 
find reassurance for the Dean at Everyone 
and a bit of advice: 

As you move into assessment at Every- 
one, i suggest you find someone to 
observe and comment on the organiza- 
tional development implications of 
what you want to do. My point is this: 
An emphasis on assessment will affect 
the way everyone functions as an or- 
ganization. You not only need help in 
the technical side of assessment but al- 
so in the nurturing of a climate charac- 
terized by: 

1. Deep concern for "results" over 
"form." 

2. Commitment to high standards. 



3. Concqmmitant interest jn helping 
students reach the standards; 

In the last analysis an emphasis on as- 
sessment is more of an attitude than a 
collection of tests, (p. 53) 

Harris' advice to the Dean^ should be 
well taken. All of the technical improvement 
possible in test making and^ administration 
will not necessarily result Jn the outcomes of 
education that we all might expect or hope to 
have happen. Taking the time to plan for the 
outcomes we expect will add enormously to 
the success of whatever evaluation or assess- 
ment colleges undertake to move closer to- 
ward their overall goals. 

Although planning models are widely 
available in written as well as software form, 
such effort often is wasted because of the lack 
of control over the actions and efforts to 
bring about desired outcomes. instiUitions 
oftentimes create elaborate, sophisticated 
plans on paper without effective management 
controls to bring about results. 

As institutions and states undertake 
plans for assessment that will yield useful 
outcomes, certain requirements of control are 
necessary: 



1 . A rational , organ ized approach to p lan- 
ning and to the choice of desired out- 
comes is essential to any accomplish- 
ment expected. 

2. Clear documented decisions on what to 
c on tro I will assist p I an n i n g an d wi 1 1 ai d 
in the accomplishment of the goals and 
objectives institutions and states set for 
themselves. These decisions should re- 
present consensus on what is viewed as 
critical in the accomplishment of the 
outcome and what forecasts trouble in 
the movement toward that goal. 

3. The senior management team at an in- 
stitution or in a state must decide on 
the method of control that will be used 
to guide and encourage progress toward 
the desired outcomes. Such decisions 
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are especially important in higher edu- 
cation where individuals are used to 
some degree of anonymity and creative 
participation rather than routine line 
responsibility. 

4. The management team must also decide 
when to control efforts toward setting 
goals and choosing assessments. These 
decisions can effectively stifle or en- 
courage the accomplishment of out- 
comes at both the institutional and 
state level. In education the decision to 
control is in many ways a matter of 
timing. 

5. Progress toward outcomes must itself 
be measured in order that all partici- 
pants may see clearly that their efforts 
are leading toward desired outcomes. 



In view of the increasing sophistication 
and availability of high quality assessment in- 
struments) the importance of planning and 
control cannot be underestimated. Outcomes 
such as those in the areas described above are 
not attained easily by even the most coherent 
software analysis package. Time and effort 
afforded to planning and control cannot help 
but improve the attitude of "assessors" and 
"assessees" but even more practically, such an 
emphasis may ensure that assessment is worth 
its cost. 
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ASSESSING VOCATIONAL COMPETENCIES 

Michael T. McCord 
Athens Technical Education Center 
(formerly Assistant Professor 
Institute of Higher Education 
The University of Georgia) 



The concept of assessment wou ld seem 
to be particularly suited to vocational educa- 
tion. First usage of the word did not emerge 
from an educational classroom, However, the 
meaning of this term began with an important 
idea for educators of all areas (Loacker, 
Cromwell, & O'Brien, 1985). From the late 
Latin, ad + sedere, the term came to mean sit- 
ting down beside someone or sitting down to- 
gether, in early usage, an assessor was one 
who sat down beside or shared another per- 
son's position. The idea of a person sharing 
with another knowledge and skills essential to 
performance has long been at the heart of vo- 
cational education. The majority of voca- 
tional instructors possess occupational experi- 
ence and are certified as skilled professionals 
within the area they have been asked to 
"share" (teach) with students, young and old* 
who desire work in that area. The vocational 
instructor is one who has been in another pier- 
son's position in business and industry and is 
now sharing that knowledge and experience 
by sitting down with another. 

As the language has evolved and words 
changed to take on differing meanings, the 
word 'assessment' and the label of assessor 
have taken on monetary connotations. The 
focus of the term has come to be on deter- 
mining the worth or Value of something, Usu- 
ally in dollars and cents or time, gold, pre- 
cious items, etc. Yet it has not left the idea 
that there has to be a skilled or expert judg- 
ment made to determine worth after some 
careful observation. 

As this paper was prepared, observa- 
tions were made of the instruction and the 
organizational structure supporting post-se- 
condary Vocational education. An impres- 
ive array of complex subjects and means for 



teaching them was discovered. Equally 
impressive and somewhat perplexing was the 
complexity of the testing and/or evaluation 
reflected in the variety of instruction en- 
countered. As a former teacher educator, my 
curiosity was aroused as to the educational 
background and experience required for certi- 
fication and licensing in vocational education. 
I was especially interested in how such a wide 
variety of skills in this instruction would be 
standardized and taught through the teacher 
preparatory institutions. Upon examining the 
Teacher Certification in Georgia handbook, I 
was pleased to find a special set of certifica- 
tion requirements for the post-secondary only 
V certificate series. 

Basic to becoming a vocational instruc- 
tor is being licensed by the professional board 
or association which sets the standards for the 
field, included in the licensing process is the 
requirement of passing the NOCTi exam _ar 
propriate to each area and the TPAI (Teacher 
Performance Assessment instrument) which is 
especially designed to assess teaching skills. 
NOCTI is the acronym for National Occupa- 
tional Competency Testing Institute, a non- 
profit organization devoted to developing and 
validating tests for determining occupational 
competency at different levels. In Georgia, 
there are over 30 of these tests approved at 
the post-secondary level as one measure of 
occupational competency. 

Also found was a listing of courses 
available at the state's teacher preparatory 
institutions which covered desirable subjects 
recommended for certification. Much to my 
dismay, there was only one course at only one 
institution which dealt in depth with the to- 
pic of testing and measurement techniques 
appropriately applied to vocational education. 
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This does not mean that vocational jnstr ac- 
tors do not receive a course In test and 
measurement principles. It does, however, 
support a contention j shall pursue In this 
paper that vocational instructors are receiving 
an ^ overemphas|s » on the use of a type of 
measurement technique which done is not 
appropriate for assessing vocational com- 
petencies. Why do J cons|der this^ Important? 
A conclusion of this paper Is L that the key 
element involved in assessing vocational com- 
petencies resides in the assessment compe- 
tency of the vocationaj infractor For 
vocational education to assure the possession 
Pf v 9catiqna I com petencies by it graduates, 
the process of assessment must become in- 
timately intertwined _ with teaching to the 
point of being considered a specbtj technique 
of instruction. One implication of this conclu- 
sion is that teacher trainers must r^ 
the JP_ ec '?'_ nature 9^ acquiring vocational 
competencies and no longer be satisfied^ to 
have the traditional means of tests and mea- 
surement techniques thrust upon vocational 
teachers as the "standard" for assessment 

This paper has the following major purposes: 

1 ) Exam ine some of the bastes of ^assess- 
ment ^nd briefly review techniques of 
assessment as they apply to vocational 
COf P Potencies, _ 

2) Present the pros and cons of^ojiduct- 

to spark dialogue among professionals, 
and 

3) Give consideration to critical questions 
and accompanying tasks which^ may 
lead to the recommendation and re- 
cognition of standards for vocational 
competencies w^j?! 1 __wp?id support 
advances in the state of the art and use 
of assessment techniques in vocational 
education. 

Why assessment of vocational competencies? 

Wentling (1980) hss compared the vo- 
cational educator's means of justifying his 
work to the sales technique employed by a 
huckster of a cure- all medicine. The huckster 



will swear by his product, will give many ex- 
amples of Its success arid produce testimony 
from someone in the audience. However, 
there Is usually some question as to how 
widely tfiis product is being used and what 
percentage of the users the success cases re- 
present. In some cases; the honesty of the 
salesperson becomes questionable. 

If broadly considered, this analogy is 
applicable to most educators. ^ ._. ^instructors 
and administrators (probab|y a higher propor- 
tion of the latter). Instruction is considered to 
be very relevant, very reliable and effective. 
Instance^ of successful former students are 
known and read|ly displayed as are statistics 
of job placement resulting from a follow-up 
study {usually short term). Yet, educators 
share a common consternation when asked to 
produce solid evidence as to the success of 
individual courses and programs. 

tlie production and use of a process for 
competency assessment can facilitate com- 
munication of program successes to the ap- 
Pl^Pr'jrt^^ucliences. Th'A tiroe °f y e a r __ ori ? 
^PP ro P_ r '?j? ??d'? rice _ tfi at readHy comes to 
mipd is the legislature. School boards and_ad- 
Yl s P r y J^ oa rds would also be better informed 
and reassured of accurate information. _Wjth!n 
the institution, administrators and faculty 
could better communicate. Even potential 
students would be impre^d^ wiS proper 
techniques for assessing competency. Of 
course, tfie results would yield to Jhe em- 
ployer an accurate picture of the perspective 
employee possessing the traits desired. 

An ad d i J onaJ j m pe tu s to devel op a be t- 
ter process for assessing vocational [competen- 
cies emerges froni several J^ent activities in 
the state, this year the establishment of a 
State Board of Post-Secon dary Vocational 
Education has resulted in the production of 
an Evaluation, Planning and Budgeting Model 
by which area institutions must conduct Jong 
range planning. This model will provide a 
statewide coordination of resources and 
means by which each institution will engage 
in program evaluation activities. As a part of 
the program evaluation activities, there will be 
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an assessment of instructional activities, . This 
is a beginning for assessing vocational 
competencies. 

The focus of competency ?ssessment in voca- 
tional education. 

The term '"vocational competency M is 
almost as broad a term as student achieve- 
ment or educational outcome. Just as with 
these termSj it is critical to define clearly what 
you are talking about before you toy to set 
about assessing it. It is important to define 
the term carefully so that it may be of utility 
for practical purposes of developing assess- 
ment, just as classifying behavior is critical to 
successful observation in traditional academic 
institutions, within vocational education it is 
important to classify behavior. 

The same domains which Bloom et. al. 
(1956) have formulated for educational a- 
chievement behavior also appiy to vocational 
education. The cognitive domain includes the 
m ost basic level s of thinking, recalling of facte 
as well as more complex levels such i_ as ; analy- 
sis, synthesis, and evaluation in vocational in- 
struction as well as in traditional academic 
instruction. Another major type of student 
learning that is critical for vocational com- 
petency is the interest and attitud in al area 
wh ich comes under the affective domain. This 
is very much what employers classify as being 
task oriented or committed to work activities. 
Perhaps the domain which holds the most 
pertinence and relevance to the assessment 
competency of vocational education is the 
performance domain. This is made up of two 
sub-domains which are psychomotor domain 
(Simpson, 1966) and perceptual domain 
(Moore, 1970). The combination of these two 
sub-domains relates to doing physical tasks 
with the use of muscle movement and a sense 
for input 

These three domains should provide 
vocational educators a means of looking at 
the job task as well as classifying vocational 
competency. The domains do help identify 
individual types of competencies in learning. 
It should be emphasized that they are inde- 
pendent But, their value should never be 
prioritized because in vocational settings it 



mayj>e crucial that an attitudinal value is just 
as Important as a performance value. Most 
performance skills require knowledge and an 
affective component for their successful com- 
pletion. For instance, a welder must be aware 
of proper tern peratu res to use. He^must use 
those proper temperatures to complete his 
work successfully but if he does not possess 
g°°d safety attitudes, he may not perform 
adequately at all; 

Within vocational I education, assess- 
ment must also be considered froin an addi- 
tional perspective raSer fihanjust domains of 
behavior and learning. Assessment should also 
be considered from a point of reference; 
particularly the two references called^ norm 
and criterion. While undergoing instruction 
and training in vocational skills^ it^may be 
^f a L? 0 A on[ lPi l _ re 1* student's performance to 
that of another student which is jiorm re- 
ferenced testing. It is critical that the major 
emphasis for the development of competency 
in vocational education be placed upon the 
student reaching criteria of performance. 

Wh He traditional [schemes ©^grading us- 
ing standardized test and performance com- 
petition jire utilized in ffie instruction that 
occurs in vocational education, this should 
not^ be the predominant mode of assuring 
competency on the part of the student. Norm 
referenced testing seems to be useful if selec- 
tion of th^most qualified student jsjmpor- 
tant or if there js a need to determine Jiow an 
individ ual or group compares to others when 
performing _a certain tok^However, when it 
comes to the assurance that a^ s tin den t pbs- 
sesses a certain vocational competence, tfieji, 
another student's evaluation has little to do 
with the perfonnance of one student L This 
should be related to a predetermined 
behavior, a standard or criterion of 
performance which employers are expecting 
new employees to exhibit 

Brief view of techniques for assessment 

Many ways exist to assess learning as it 
occurs within vocational programs. To at- 
tempt this would go beyond the scope of this 
paper. For a more detailed coverage, publica- 
tions by Erickson and Wentling (1976), Cross 
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(1973), and [ Bloom, Hastings, and Madaus 
|j 971) are referenced. First, let as consider 
the assessment of performance since it is an 
activity jn [which all vocational educators have 
participated. Prior to forma! education, the 
performance^f apprentices was assessed by 
m»ter craftemen. jn tarn, today, vocational 
instructors have been certified in their pro- 
fession^They assess student pefformahce by 
many means, sometimes intertwining the as- 
sessment spontaneously with instruction. Al- 
^ost^al^ys, performance is^as^ssed by first 
observing and then jwa[uating Jfie process or 
the product of die act thatjs being perform- 
ed. Providing feedback to die student usually 
regards the quality or the nature of their act 

There are three Basic types of assess- 
ment used for performance. There js 1) some 
type of Identification test, 2) a simulator or 
work sample, and finally 3) the employer's 
survey^ Since the employer's survey is least 
likely to be readily available to Jhe instructor 
as they are teaching, HI expound briefly upon 
the f rst two. The identification test is the 
most basic type of performance test. It mea- 
sures an individual's capabilities using know- 
ledge and sensory input to identify mater^'s, 
tools, equipment, components, and other job 
related items; This can Be as Basic as asking 
students to jdentify wood samples bras com- 
plex as asking students to identify auto parts, 
their wear or deficiency, the cause of the de- 
ficiency and tfie recommendation for repair. 

The second major type of performance 
test includes tiie simulation and work sample 
tests. When a jvork sample test is employed, 
there is the administration of a control test 
under the actual I conditions of the work situa- 
tion; .Usually the stodents are required to use 
the same procedures that would be employed 
or^the job. Simulation tests, however, are de- 
signed to replicate the work situation by using 
specialized equipment or making modification 
ln_ existing equipment and imagine they were 
on tiie job. in many vocational educational 
laboratories a project method [of instruction 
uses a form of the work sample test In this 
situation, a one-to-one relationship exists bet- 
ween examiner and student. The content of 
work sample and simulation tests can vary 



from very simple, immediate reactions to 
very complex tasks such as those used in a 
medical laboratory. 

The category that is probably second of 
importance to use in vocational education is 
thai of affective behavior. Affective compe- 
tency can be categorized into four areas for 
assessment purposes: interests, attitudes, 
values and appreciations (Tyler, 1973). As- 
sessment in each of these catagories is diffi- 
cult for several reasons. The technology for 
testing the affective domain is somewhat 
primitive, especially when compared to the 
technology for testing and measurement in 
the performance and cognitive domain. It is 
also very difficult to measure the Impact of 
instruction on affective behavior, because the 
effect of instruction is often long term, 
possibly not being exhibited for months or 
years after instruction. By exhibiting pride, 
the vocational instructor may be teaching 
pride in work. This is hot measured or ob- 
served until a student has entered the world 
of work and had the opportunity to exhibit 
pride which was observed and acquired. 

While these difficulties are recognized, 
they should hoi be ah excuse for failure to at- 
tempt assessing competencies. Instead they 
should be considered a challenge and provide 
a goal for advancing competency assessment 
within the affective domain. As ah example, 
within all vocational programs there Is a con- 
tent area of safety which should be taken as a 
common area for which standards of affective 
behavior can be established and therefore as- 
sessed. This may establish a model for as- 
sessing other affective competencies in 
vocational education. 

There are r Jier ways of assessing af- 
fective competencies. There are generalized 
and direct observations by the instructors, 
interviews, questionnaires, inventories, pro- 
jection techniques and unobstrusive measures, 
all of which have been employed to assess 
various aspects of the affective domain. How- 
ever, in vocational education we do hoi seek 
to go onto tiie depth of ohe^s perebnality to 
measure affective competency. Therefore, 
direct observation is usually appropriate. This 
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involves recording while viewing a student's 
behavior following a stimulus. An example is 
the nursing student's reaction tc a patient's 
criticism which can be observed and judged as 
assessment of interpersonal skills. 

Probably the domain which has re- 
ceived the most attention in vocational pro- 
grams is the assessment of cognitive compe- 
tencies. The reason for this is primarily due to 
the fact that performance objectives and test 
items stated as cognitive skills are easier to 
prepare than for other domains. It is also pre- 
dominantly the traditional means by which 
achievement has been judged. Consequently, 
procedures and techniques for assessing cogni- 
tive skills are the most highly developed and 
emphasized. If vocational education is to re- 
assure employers that vocational competen- 
cies desired have been taught, surely jt is 
necessary for assessment procedures to have 
evaluated the student for more than just cog- 
nitive achievement This is hot to discourage 
the use of cognitive tests, it is merely pointing 
out that instructors should not be bound by 
the tradition of putting importance on the use 
of cognitive tests more than those of perfor- 
mance or affective competencies within 
vocational education. 

Pros and cons of assessing vocational 
competencies. 

Initially, it would appear that the most 
important advantage in assessing vocational 
competencies is to assure the employer of 
what he is acquiring in a student who gra- 
duates from a vocational institution. How- 
ever, I would argue that this is further down 
the line of important reasons. In my opinion, 
it would appear that the most important 
reason for assessing vocational competencies 
and having a process which does assess voca- 
tional competencies is to enable the im- 
provement of instruction. This is to benefit 
student and teacher interaction. If students 
failed to become competent, in most cases 
there has been some flaw in the instructional 
process. If emphasis is given to the assessment 
process, the instructor becomes attuned when 
weaknesses develop, This provision of feed- 
back to teachers about areas of instruction 



needing. further emphasis can provide a major 
means for continually upgrading instruction. 
Thus, assessment results can enhance instruc- 
tion. When results are returned to students, it 
can be demonstrated to the student that their 
competency is growing or that their learning 
is progressing. For those students who are hot 
progressing, a knowledge of assessment results 
can be a basis to establish additional or fur- 
ther study. 

Another advantage of using ah assess- 
ment process to establish competency is that 
it insures quality control of instruction over 
time. When standardized achievement mea- 
sures are used by the instructor year round, 
the results of instruments can be compared 
across the classes to determine if instruction is 
consistent O r . the other hand, when criterion 
referenced measures are em p loved, there is a 
constant flow to the student of strengths and 
weaknesses. Thus, a constant recognition by 
the teacher of where the student needs to up^ 
grade. This is one area which could be of 
greatest benefit: establish criterion reference 
testing procedures for teachers to utilize in 
assessing vocational competency.. Levels of 
competencyjcould be established. Rather than 
comparing to other students or norms, there 
would be identifying levels of competency. 

Of course, consideration must be given 
to what are some of the disadvantages of 
assessing competencies. One of the first and 
probably the most obvious (arid available as 
an excuse), is that by establishing criteria to 
which students must strive instructors may 
limit their teaching to the content or the 
proverbial "teach ing to the test 99 If the test is 
based upon the content needed, that may hot 
be so terrible; however, it may have another 
effect of "minimal" instructors teaching to 
the minimum criteria. Another disadvantage is 
related to the underdevelopment of assess- 
ment technology. Compared to procedures 
used in the physical sciences, the measure- 
ment of learning, in general, is in its infancy 
because it lacks the precision thai can be 
attained in the physical sciences, as already 
mentioned, probably the assessment of 
cognitive leaning is the most advanced of all 
assessment techniques. But assessment of 
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affective and performance behavior is crucial 
to Vocational education. One of the reasons 
cognitive achievement is so readily utilized in 
Vd national instruction is there are a number of 
aw able standardized tests which have under- 
gone extensive revision. 

Another drawback is the time assess- 
ment procedures can consume, if Vocational 
competency is to be assessed correctly, it is 
going to require both student time and in- 
structor time interacting in ways Which may 
displace some traditional instructional acti- 
vity. There will be a heed for the instructor to 
document the assessment process arid this will 
require the preparation of recording the as- 
sessment procedure. Another and probably 
final disadvantage of assessment relates to its 
focus, if we focus too strongly in assessment 
on the learning that takes place, then top 
much emphasis will be on achievement arid 
hot on instruction. It would be crucial for as- 
sessment procedures to hot only indicate 
when low achievement occurs but for the 
supervisor to be able to identify the reason 
for the problems. 

Regardless of the disadvantages, the 
pros for assessing competencies outweigh the 
cons. Recognition of shortcomings should 
provide impetus for devising ways; of minimi- 
zing thesr effects in the assessment of voca- 
tional competency. 

Critical questions about assessing competen- 
cies in vocational education. 

There are several questions involved in 
the process of designing assessment proce- 
dures for vocational competencies. Along 
v/th each question an activity is necessitated 
to produce an answer and establish the credi- 
bility of that answer. The first question has to 
be (Q1) what should be the competencies ac- 
quired? There are several means to ascertain 
the answer to this question, but in order for 
the answers to be considered credible, there 
must be an accompanying task of (T1) asses- 
sing the Validity {appropriate and adequate) 
of identified competencies. Upon the stu- 
dent's completion of the program of study, 
(Q2) what were the competencies actually 



acquired? Here standardized tests can give lis 
information of the acquired knowledge. In or- 
der to really ascertain whether or hot we have 
done a good job of measuring the acquired 
competencies, there must be the ta$k of (T2) 
testing the validity of measurement techni- 
ques with interviews, observations, simula- 
tions, and the use made of test results during 
the student tenure in the program of study. 
The next question is (Q3) how well did the 
instruction work? The accompanying activity 
on task is (T3) assessing the instructional 
training process as it was planned and as it 
was actually executed. 

The above three questions relate to the 
acquisition of the competency and to what 
happens within the local institution. However, 
there is still a question of (Q4) do the gra- 
duates apply their competencies? This 
necessitates an accompanying task of assessing 
the extent of competency application in 
actual Job/occupational performance situa- 
tions. Even therm it cannot be assumed that 
we are finished with the student We must go 
on to ask the question of (QS) How long the 
competency lasts? This necessitates the ac- 
companying task of (T5) assessing the en- 
durance of competency, in performance 
situations. That is followed by the even more 
broad question of (Q6) what is the 
socioeconomic impact? This brings to the 
task not only (T6) assessing the benefits asso- 
ciated wixii vocational competency but the 
cost. Any tame we seek the socioeconomic 
benefit, we must take into account what it 
cost to produce those benefits. Thus, there 
has to be some means of ongoing cost cap- 
turing procedures. At the same time, there 
must be follow-up study of long term impact 
associated with the student's career. The final 
question brings us back to identifying compe- 
tencies in the first place, just a bit prior to 
asking that question is another. . . (Q7) which 
competencies do we need most? The accom- 
panying task is one of (T7) prioritizing voca- 
tional education efforts in terms of the 
relative value of competencies to the public. 
If the market is flooded, if the technology is 
changing such that bur production of agricul- 
tural engineers or technicians or welders or 
machine shop specialists is no longer highly 
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valued, then there must be a change in priori- 
ties to de-emphasize these and to emphasize 
other skills thai have come into existence. 

A point which needs to be considered 
in determining an answer to what is the socio- 
economic impact goes beyond the definition 
of program success and examines multi- 
dimensional indicators of the worth of curri- 
cular content. Judgment of the w.>rtH bf a 
program should embrace such matters as 1) 
congruence. with an individual's career objec- 
tives, 2) differences in learning style^ 3) hpri- 
vocational purposes, 4) psychological satisfac- 
tion, 5) influence on eventMal educational 
attainment, 6) short and long term economic 
benefits to the individual, 7) social efficien- 
cies in work, place, 8) ability to fulfill the 
aspirations of persons with special needs, e.g. 
a physical handicap, and 9} the basic cost 
effectiveness (Grasso and Shea, 1979). 

Conclusion ar.d recommendation. 

The assessment of vocational compe- 
tency is a complex task. It will require a com- 
plex approach but it should be a straightfor- 
ward approach beginning at the top level. The 
establishment this year of a State Board of 
Post-Secondary Vocational Education is the 
beginning of an autonomy and authority 
which will allow the development of assess- 
ment particularly suited to vocational educa- 
tion. The guidance necessary for Local 
institutions has begun in the form of an 
evaluation, planning and budgeting model 
(State Board of Post-Secondary Education, 
1985). This model contains two phases of 
program assessment The first is based upon 
enrollment, graduates and placement. The 
second involves a higher degree of student 
evaluation, employer evaluation, assessment 
of program facilities, assessment of the equip- 
ment and instructional materials, assessment 
of the instructional process and assessment of 
business support. The Board is to be com- 
mended for taking fist arid accurate actions in 
a direction that will prove beneficial to the 
effective development of vocational education 
at the post-secondary level in the state of 
Georgia. 



There does exist a heed for establishing 
standards by which efforts to measure voca- 
tional competencies should seem to measure 
what they purport to measure; that is, they 
should have face validity. Within psycho- 
metric circles, face validity is often accused of 
having a detrimental impact because it a- 
rpuses a response set of faking tendencies. 
However, within vocational (ducat ion if a 
measure does hot have substantial face valid- 
ity to the l instructor and to the employer, 
then it will hot receive credibility. Given the 
basic politics and purposes of assessing voca- 
tional competency, face validity is necessary 
for assessment to be> taken seriously. . . even if 
on pyschometric grounds it is not sufficient 

A second standard is that vocational 
competency assessment must have a generic 
component applicable to different levels 
across different fields. While the previous 
standard encourages assessment procedures 
that are individually suited for each field, this 
standard establishes the necessity of bringing 
together some convergence, consistency, or 
uniform quality that can be expected of all 
graduates from vocational education institu- 
te us. Here again the State Board is to be 
cbriimehded._ The work of the Standards 
Committee (1986) has already recognized two 
categories of program standards, those which 
apply specific occupational programs and 
generic standards which apply to all programs. 
Under the definition of generic standards this 
committee has indicated components should 
be the basic skills of communication, compu- 
tational, and employability skills, each of 
which should include pertinent computer 
literacy skills. 

A third standard for the measure of 
vocational competency is that they should be 
based on criteria that are public. Thus, differ- 
ent evaluatcrs will make approximately the 
same assessment of the same student's perfor- 
mance. What is referred to here is the neces- 
sity that criteria be established <>h which 
observers can agree but at the same tiriie 
should also be very, explicit and should be 
known in advance. Students should have ac- 
cess to the criteria towards which they are 
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developing. When Jhese criteria are public, 
then the assessment can be questioned by 
students and defended by faculty in a 
straightforward fashion that dampens any 
suspicions of favoritism or futility. 

A fourth standard is thaL measures 
should have educational validity. The assess* 
merit procedure should be able to demon- 
strate that students at the end of the program 
will score higher than themselves or other 
students when beginning the program. While 
this seems obvious, it is often neglected and 
treated casually in practice. It implies several 
things, one of which is that there will be a 
careful coordination between curriculum de- 
sign and the assessment of competence. This 
standard would emphasize ah importance of 
criterion referenced rather than standardized 
testing. This would n/ean that most tradition- 
al measures of ability are not very useful as a 
comprehensive measure of vocational compe- 
tency. 

The fifth standard that I would impose 
upon assessment of vocational competence is 
thai they should be demonstrably relevant to 
performance after graduation. Here the issue 
is hot so much as to whether the assessment is 
relevant but the basis for claiming relevancy 
and the kind of evidence brought forward to 
back up the claim. Accompanying these re- 
commended standards should be activities to 
support assessment in the form of systematic 
attempts to advance a technology of assessing 
vocational competency. I understand current- 
ly a test bank is being developed for all voca- 
tional/occupational areas by the VTEC pro- 
gram with the cooperation of the Southern 
Association. While this is commendable, 
inherent within these standards is the recom- 
mendation that assessment procedures in 
vocational programs be extended and go 
beyond cognitive learning. An implication 
would be thai vocational instructors will need 
to expand their knowledge of performance 
and affective assessment techniques through 
formal and informal staff development 
activity. 

A final recommendation is tiiat there 
be ah emphasis oh using a variety of techni- 



ques to supplement the assessment of compe- 
tency beyond the student's tenure. Evaluative 
techniques such as a long term follow-up 
study, employer surveys and advisory com- 
mittees as evaluating teams should be Utilized. 
The results of these assessment efforts should 
be used to facilitate the understanding and 
correction of program deficiencies on the part 
of instructors and administrators. 

In closing, I would assert that the 
development and definition of assessment has 
preceded along lines through history to a 
point where for vocational educators it means 
more than evsr a sitting down together with 
the learner co assure the acquisition of skills 
for a bright future filled with opportunity. 
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STATEWIDE NEEDS ASSESSMENT FOR HEALTH PERSONNEL 



Libby V, Morris 
Institute of Higher Education 
The University of Georgia 

I am pleased to be able to present to you 1983 there were 191 general hospitals with 

this afternoon an overview of the "Statewide 33,468 beds and over one million admissions. 

Needs Assessment for Health_ Personnel" Added to this extensive delivery system 

currently being conducted by Dr. Cameron was Georgia's 326 nursing homes with 33,730 

Fincher and myself at the Institute of Higher beds with a 94.0 percent occupancy rate. 
Education. This study is funded by the 

Georgia Student Finance Authority, the These impressive growth statistics 

agency which awards the service cancelable represent a technologically advanced system 

loans to students in the health professions. of health cafe with equally startling health 

First, I would like to give a very broad and care costs. Consequently, there Has ensued a 

general description of the health care delivery lively debate and mixed reactions as to how 

system in the United States and Georgia, and to contain health care costs and deliver health 

then I would like to touch upon the major care services. Several of the reactions to 

points of this study within that context contain health care costs are as follows: 



Health care in the United States is a major 
business and a national concern, in 1982 
health care expenditures totaled 322.4 billion 
and comprised 10.5 percent of the gross 
national product The average cost that year 
for health care was $1365 per person. During 
the 1960s and 70s, jobs in doctors' and 
dentists' offices more than doubled! having 
climbed to 1.5 million in 1982. At the same 
time hospital employment tripled growing by 
5.1 percent per year to three million jobs. 
Even 5n 1979-82, when total employment was 
virtually unchanged, employment in health 
occupations continued to grow. 

This tremendous growth was the result of 
many factors. Private medical insurance 
increased tc cover greater numbers of people. 
The federal government in 1965 introduced 
Medicare and Medicaid to reach the indigent, 
disabled and elderly. Technological advances 
soared creating job openings for previously 
unheard of technicians and technologists. 

During the 1960s and 70s, accompanying 
this growth in employment and technology 
was an expansion of facilities, in 1873 there 
were 178 hospitals in the United States, in 
1980 that number had grown to over 7000 
hospitals nationwide. In Georgia alone in 



1. Prospective Reimbursement Plan 

The prospective reimbursement plan is the 
federal government's plan for holding 
Medicare costs down by reimbursing 
hospitals at a fixed rate for 468 defined 
diagnostic groupings. 



2. Health Maintenance Organizations (HMO) 



3. 



HMOs are designed to control costs and to 
emphasize good health habits. HMOs 
contract with an enrolled population for 
physician and hospital services at a fixed, 
prepaid rate. Metro Atlanta Has six HMOs 
with an estimated 1 80,000 members. 

Preferred Provider Organizations (PPO) 

PPOs are one of the newest concepts to 
control health care costs. In PPOs, a large 
business contracts with certain doctors 
arid hospitals to provide health care 
service to its employees at a reduced fee 
for service. The fees may be as much as 
15-20 percent below the usual and cus- 
tomary charge. 



Some of the delivery system changes have 
included the entry of "for-profit" hospitals 
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owned by national chains, and the establtsh- 
men of birthing centers, mobile diagnostic 
units, home health care agencies and free- 
standing emergency centers. 

In one year, 1982, freestanding emer- 
gency centers increased from _ 260 to 600, 
and the National Association of Freestanding 
Emergency Centers (NAFEC) currently 
estimates 1800 such centers. It is predicted 
that there wfll be 4500 by 1990. Atlanta 
currently has 23 centers, all privately owned 
and for profit 

The newest market in health care, how- 
ever, is in outpatient services. In 1983 
Georgia had 70 home health agencies serving 
all 159 counties with services ranging from 
professional health care such as physical 
therapy to housekeeping aid. A drive along 
any major street will reflect the proliferation 
of durable medical goods and services in the 
home health care market 

this brief description of the medical 
marketplace is hot to imply that the pro- 
liferation of services has reached every comer 
of Georgia or that competition has spurted 
sharp declines in health care costs. For 
example, "less than 10 percent of Georgia's 
physicians practice in non-metropolitan 
areas." Additionally, there are 14 counties in 
Georgia where the primary care providers do 
not accept Medicare or Medicaid, Also, in 
1984 health care expenditures climbed to 366 
billion dollars, surpassing defense, education, 
food and beverage, consumer goods, financial 
institutions and agriculture. 

These descriptions are illustrative but not 
exhaustive of changes in the health care 
system in the 1980s; yet, as a result of these 
and other measures, hospital stays are shorter 
and utilization is down. Last year, one in 
three hospital beds was vacant at any one 
time, and as a result, hospitals cut 73,000 
employees from their staffs. The questions 
then are: How will these changes in the 
marketplace affect bur future supply and 
demand of health care professionals? How is 
it impacting on the current supply and 
demand of professionals? What do these 



changes mean to our educational system-its 
applicants, enrollments, its growth or demise 
in certain areas? 

While changes were occurring in the 
deliver/ end of healtii care, changes were also 
happening in the supply end of education. 
Nursing supply can serve as one brief 
example. The number of registered nurses 
actively practicing quadrupled between 1950 
and 1980 from 335,060 to 1.2 million. 
During that time, traimng programs shifted in 
type and location: college and university 
programs at the baccalaureate level ^grew from 
195 to 368. Associate degree programs grew 
from 18 in 1955-56 to 767 prognnm Jri 
1980-81. The traditional training programs 
for nurses, the hospital-based diploma pro- 
grams, decreased from 967 to 31 1 in 1980-81. 

As can be seen from the foregoing 
description, the health field is changing, and 
at the center of this change are the educa- 
tional programs, the patients, and the health 
care professionals. For without patients, 
there is rib demand for health care services, 
arid without an educated arid ready supply of 
professionals, expressed demand cannot be 
met 

It is the study of supply arid demand Sri 
Georgia that is being conducted by the 
Institute of Higher Education. The objectives 
of the study are as follows: 

1. to survey the educational arid training 
programs for health professionals in 
Georgia-their growth, availability and 
productivity; 

2. to inveritory^the traditional and emerging 
sites of employment for health pro- 
fessionals in Georgia; 

3. to assess Georgia's current ^ supply and 
overall need for profess]onal]y trained 
personnel in the health care field; 

4. to project the future supply and future 
demand for health care professionals in 
Georgia; 
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5. to assess how supply and demand might 
be brought into balance through assis- 
tance to students through service can- 
celable loans; 

6. to make recommendations concerning the 
education of health related personnel and 
the way in which public resources might 
better be used to improve health care in 
Georgia. 

This study began in the Fall of 1985 and 
will be completed Fall, 1986. Under consid- 
eration are approximately 25 careers 
including dentistry, nursing, pharmacy, respi- 
ratory therapy, occupational and physical 
therapy, surgical technology and podiatry, to 
name a few. 

Rather than describe manpower method- 
ologies and instrumentation for this study, j 
would like to review the study by addressing 
its three major components: supply, demand 
and projections of future supply and demand. 

Supply 

Supply can be defined in many ways but 
genera 1 '** refers to the actual head count of 
persons actively working in a field. Supply 
can be influenced by a number of variables 
that are difficult to measure: immigration 
(i.e., the number of eligible workers moving 
into the state); outmigration (i.e., the number 
of eligible workers leaving the state); part- 
time workers; cross-over workers (i.e., health 
care workers of one occupational type per- 
forming the service of another occupational 
type); licensure status for an occupation; the 
educational systems 9 output in the form of 
students and graduates; and the demographics 
of the current supply . 

To -ddress the problem of establishing the 
current supply in Georgia, it is possible to go 
to a number of sources, but as will be quickly 
seen, each data source has its problems. 

1. State Examining Boards 

The state examining boards regulate the 
practice of certain occupations in Georgia 
through an examination and licensure 



process. Thirteen of the occupations 
under consideration are regulated through 
those boards with biennial requirements 
for re-licensure. The problem with this 
measure of supply can be stated by the 
phrase "once licensed, always licensed." 
The State Boards do hot regularly, ;f at 
all, collect information on the practice 
status or location of those persons holding 
licenses. As a result, the supply according 
to licensure status is greatly overstated in 
most cases. 

2. Educational Programs 

Educational and training programs are, of 
course, the major source of supply. To 
determine the number added to the 
potential supply each year by the educa- 
tional programs, all of the training 
programs in the state will be surveyed as 
to their program's productivity (i.e., 
capacity, number of applicants by 
program, number accepted, number 
enrolled, number graduated). The schools 
will also be surveyed for plans of re- 
duction, expansion, elimination or 
addition of programs. 

For the University System, this process 
will be more easily accomplished by re- 
ferring to the data on programs and 
students at the Board of Regents. More 
difficulty attends securing the same infor- 
mation from the many private colleges, 
proprietary schools, vocational-technical 
schools, and hospital programs in Georgia,, 
For example, there are currently 14 
hospitals in Georgia offering one or more 
health professions programs accredited by 
the American Medical Association's 
Committee on Allied Health Education 
and Accreditation. There are also other 
programs in Georgia hot accredited by a 
recognized accrediting body, but these 
programs, too, add to the supply. 
Developing a comprehensive list of who 
teaches what, where, to whom in Georgia 
is not a simple task. 

3. National and State Professional 
Associations 
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Supply may also be addressed through 
contact with the national and state pro- 
fessional associations. Those older, more 
traditional, occupations have stronger 
professional associations with more cur- 
rent and reliable data on programs and 
students. The newer occupations often 
have Very short training periods, multiple 
unaccredited programs and very little 
data. Even with the older professional 
associations, supply is usually addressed 
according to educational supply rather 
than working supply. As we all know, 
everyone educated in an occupation does 
not choose to work in that occupation. 
For example, some authorities in nursing 
estimate a 30 percent attrition from that 
field. 

Since many of the occupations included 
in this study are classified as Allied 
Health! and are relatively new on the 
health scene, even jess is known about 
their graduates, attrition, employment 
and productivity. In fact, sixteen of the 
twenty-four occupational areas for which 
the Committee on Allied Health Edu- 
cation and Accreditation (CAHEA) ac- 
credits programs have been added since 
1968. 

4. State and Federal Agencies 

Lastly, for supply estimates the state and 
federal agencies may be consulted. One 
may turn to the Georgia Department of 
Labor, although that agency has Slot been 
funded since 1982 to conduct specific 
"manpower" studies by occupation. One 
may aho go to the Federal Bureau of 
Hentr'i Professions and consult their 
reports which, of course, have a consider- 
able time lag in their data, the State 
Health Planning Agency also may be con- 
tacted for information on facilities and 
distribution of high technology equip- 
ment; however, personnel distribution, 
especially in allied health, (i.e., work- 
force studies) is not a major priority for 
that agency. 



In summary, overall provider to popu- 
lation supply in Georgia is hot easily deter- 
mined! and it becomes even more complicated 
when problems of geographic maldistribution, 
institutional maldistribution of personnel and 
areas or shortages are added to the concerns 
being addressed. Even with these problems, 
supply can and must be established to enable 
our educational programs and funding 
agencies to have the information they heed to 
make sound decisions about resource 
allocation. 

Demand 

Demand refers to those health care 
services a population requires, uses, and/or is 
willing to pay for over time. Demand can be 
assessed by such measures as manpower to 
population ratios, heed-based figures, 
demand-based figures or professional judg- 
ment estimates. Many factors, once the 
methodological approach is selected, then act 
on demand: the age of the population; the 
income and educational level of the target 
population; environmental conditions arid the 
accessibility of health services. jh Gedrgia^ we 
fmd_that in urban counties, 13.6 percent of 
the population fall below the federally- 
defined poverty level, jh rural areas, the 
percentage is significantly higher at 22.3 
percent The economic difference in the 
population affects demand in rural versus 
urban counties. "Need" may be the same, 
but the economics of poverty create a 
different "demand" for services. 

To determine demand as defined by 
employment, a survey will be made of 
Georgia's health care employment opportu- 
nities. Included in the survey will be all of 
the state's nursing homes, hospitals, home 
health agencies, public health services, and 
private offices, clinics and laboratories. One 
of the larger tasks is to identify all of the 
users or employers of health personnel. 

The employment survey will seek to 
identify current levels of employment, vacan- 
cies, turnover rates and the plans for addition 
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and deletion of health _ K.rvlces. TH'^A 1 ! 1 ?! 1 ? 1 ^ 

mation will be combined with data collected 
from several annual surveys such as the joint 
Annual Hospital Survey and the Ambulatory 
Surgical Services Questionnaire. Establishing 
demand is crucial to any workforce study, for 
educating individuals in jobs that are over- 
supplied and in decline is too costly in federal 
and state dollars, too costly in university and 
college resources and too costly in human 
time. 

Future Supply and Demand 

The third area of concern is the future 
supply and demand of health care prof* 
fessionals. the future supply and demand 
will be established by considering several 
variables: current trends in supply and 
demand, current and projected population 
data for the state, technological advances and 
changes and the profile of Georgia's future. 

Between 1980 and ! 1984, Georgia was the 
third fastest growing state in i the nation and 
over half of that growth was attributed to 
Emigration. Between 1980 and 2000, 
Georgia is predicted to add two^ million more 
people to its population. Currently, over 1 0 
percent of Georgia's population is over 65, 
and that percent is expected to increase 
rapidly as Georgia becomes the second fastest 
growing retirement state in the South. This 
projected growth in the population in Georgia 
along with its concurrent aging will make new 
demands on Georgia's health care system and 
its educational programs. 

If Georgia is to meet the health care 
requirements of its citizens in the future, it 
must know where it stands today, where it 
wishes to go in the future, and how she will 
get there. We hope this study will provide 
answers to some of those questions and will 
be of help on the way to the future. 
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ASSESSING THE NEW STANDARDS FOR 
HIGH SCHOOL GRADUATION AND COLLEGE ADMISSIONS 

Nathaniel Pugh, jr. 
Assistant to the President 
Georgia Southern College 



Introduction 

Good afternoon, it's an honor to share 
with you my thoughts on assessment of high 
school graduation requirements and new 
college admissions standards. 

There is a concern in the nation by the 
public with the quality of education received 
by students who graduate from our public 
high schools and the academic preparedness 
of our youth who are being admitted to state 
universities and colleges. 

In 1976 [1], Peter W. brought action 
against a San Francisco school district in the 
Superior Court claiming negligence and inten- 
tional misrepresentation which deprived him 
of basic academic skills. He alleged he was 
able to read at only the fifth grade level when 
he graduated, and this was not adequate to 
enable him to enter the world of work. Two 
years later [2] a student brought action 
<*g?jnst a New York school district to recover 
$5,000,000.00 damages for his academic 
deficiencies. He claimed he was permitted to 
graduate although he had failed several 
subjects and lacked basic reading and writing 
skills. 

While these suits were not successful 
for the plaintiffs, the acci nations of poor 
public school performance were well reported 
by the media* Public schools are being asked 
by the public to account for the quality of 
their products. Thus, the significance of grad- 
uation requirements come into focus. Gradua- 
tion, as the capstone of secondary education 
logically should reflect some central priorities 
of schooling. 

At a public hearing on excellence in 
education [3] , Dr. Gary Jones cited some 
rather interesting data regarding the academic 
preparedness of American youth who are 



being admitted to our postsecondary institu- 
tions, he stated: 

A study done at the University of 
California found that between 1977 
and 1980 only half of 50,000 students 
could demonstrate reading and writing 
skills necessary for college level courses. 
At Kent State University, 25 percent of 
entering freshmen left school after two 
years of below average work. 

Georgia's 33 institutions of higher 
education spent more than six million 
dollars in 1981 on remedial training. At 
Ohio State University, 42 percent of 
entering freshmen in 1981 were requir- 
ed to take at least one remedial course 
in English or mathematics. Ohio's total 
remedial bill in 1981 was in excess of 
10 million dollars. 

One-half of City University of New York's 
170,000 students required special help and 
remedial education at a cost of 33 million dol- 
lars annually. 

You will note that this problem is not 
confined to any one region of the United 
States. Many observers feel the problem lies 
with a decade-long decline in postsecondary 
standards. Rather than engage in the over- 
used technique of "pointing the finger" at the 
the other sectors of education! namely} 
secondary and presecondary, j believe and 
support the second recommendation of the 
National Commission on Excellence in 
Education's report, "A Nation at Risk: The 
imperative for Educational Reform", that 
stated: 

"We recommend that schools, colleges, 
and universities adopt more rigorous and 
measurable standards, and higher 
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expectations for academic performance 
and student conduct, and that four-year 
colleges and universities raise their 
requirements for admission. This will 
help students do their best educationally 
with challenging materials in an environ- 
ment that supports learning and authentic 
accomplishment" (1983, p. 27) 

Assessing The New Standards For High 
School Graduation 

Until about 1976, the two standards 
mandated for high school graduation were 
attendance and credit requirements in a 
variety of specified and unspecified courses. 

Beginning in the mid-seventies, the use of 
an additional or new standard was imposed by 
states for high school graduation to assess 
academic performance based upon verified 
competencies. 

Two of the three standards for high 
school graduation that have been reviewed 
recently in the literature are credit and course 
requirements and the level of academic per- 
formance. 



A national survey [4] of states revealed a 
wide Variance in the number of credits 
required for graduation. For example, four 
states had no requirements; one state required 
only a one-semester course in government; 
and one state required only one year of study 
in history/government One state required 16 
units, but did not specify the units. Most 
states required credit in the areas of English, 
mathematics, and science. _ The modes for 
these subject areas were: English, 25 states 
required more than three units; mathematics, 
23 states required one Unit; and science, 27 
states required one unit. Eighteen states 
reported that changes in graduation require- 
ments were under consideration. Twenty-one 
states indicated that minimum competency 
tests were required for graduation. Many 
local boards of education have imposed the 
minimum competency test requirements 
without state mandate. 

the graduation requirements in the State 



of Georgia mandates attendance! earn at least 
21 Carnegie units of credft, with a total of 13 
Carnegie units that must be taken in the core 
curriculum; English language arts 4, mathe- 
matics 2, science 2, social studies 3, American 
studies area 1, health) safety and physical 
education 1, computer technology and/or fine 
arts and/or vocational education and locally 
required or elective units 8, and passing the 
Georgia High School Basic Skills tests. The 
high school graduation requirements in the 
State of Georgia are consistent with most 
states. 

The major policy issues that will be used 
to assess high school graduation requirements 
are: legal, psychometric and educational. 

The Legal issues 

The implementation of Florida's Edu- 
cational Accountability Act of 1976 resulted 
in a class action suit that has become a pre- 
cedent setting case known as Debra P. v. 
Turlington [5j. The suit alleged that the 
testing program violated the Fourteenth 
Amendment on four counts, namely that: (1 ) 
the test was culturally biased, (2) implemen- 
tation of the program denied diplomas with- 
out sufficient notice in time to prepare for 
the examination! and (3) the testing program 
was a device for resegregating Florida's public 
schools since those failing the test were placed 
in remedial classes that tended to contain 
more blacks than whites, (4) the test may 
have covered matters not taught in the 
schools of the State. 

In our own State of Georgia, students 
filed a suite known as Wells y. Banks, to 
enjoin the Tattnall County School District 
from imposing the minimum competency test 
as a prerequisite for graduation. The Georgia 
Court of Appeals upheld the school district's 
right to promulgate policy reasonably related 
to the state's interest in educating youth [6]. 

_ Following this lawsuit in another case in 
Tattnall County, Anderson __v. Banks, a 
Georgra court ruled that the diploma policy, 
per se, did not violate the Fourteenth 
Amendment on the basis of racial 
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discrimination. However* considering the past 
de jure history of a segregated system, and ?n 
unfair tracking system whicK perpetuated It, 
the discriminatory impact of the policy did 
violate the plaintiffs constitutional rights. 
However, the diploma policy cannot be cqri- 
stitutionally imposed on those who attended 
classes in the dual system. 

A review of the literature on legal issues 
and the use of minimum competency tests to 
satisfy graduation requirements indicate that 
there are five major legal problems that states 
and local school districts could face; they 
are: educational malpractice, racial discrim- 
ination, adequate prior notice* the match 
between the test and instruction, and the 
psychometric properties of the tests. 

Psychometric Issues 

Minimum competency tests used by most 
states ha^e established stahdifds of perfor- 
mance that use criterion-referenced tests. 
Criterion-referenced means that the student's 
performance is jneasured by his or her 
mastery of certain skills or subject matter 
rather than by comparison with the perfor- 
mance of other students taking the same or 
comparable tests. 

The setting of standards for mandated 
minimal competency requirements is one of 
the more important and controversial pro- 
blems confronting the entire educational 
community, and it has special salience for 
measurement specialists. 

A standard often is assumed to have been 
established 1 then is used as the starting 
point to determine whether an examinee 
should be classified above or below the 
standard. With some exceptions [7*8? little 
attention has been directed to the questions 
of where the standards come from, who 
established them, and what procedures are 
used to set them. 

The procedure used to establish the cutoff 
scores or standards for the Georgia High 
School Basic Skills Test was a modified 
Angoff/Nedelsky procedure. The modifi- 



cation was recommended by jaeger of the 
University of North Carolina at Greensboro; 

In the literature five models are used as 
methods to establish standards of perfor- 
mance for minimum competency tests. The 
models are: judgmental, combinational, 
empirical, bayesian and decision theoretic. 
There are seven major procedures that com- 
prise judgmental models (Nedelsky, modified 
Nedelskly, Angoff, modified Angoff, Ebel and 
Jaeger). 

These five major procedures that are used 
to set standards of performance are described 
as continuum rather than state models. The 
basic difference between these models has to 
do with the underlying assumption made 
about ability. Continuum mod^ view ability 
as continuously distributed attributes with 
upper and lower boundaries and an individ- 
ual's performance established at some point 
on a continuum. State models view abilities 
as an all or none proposition, either you can 
do something or you cannot 

For minimum competency tests standards 
are set arbitrarily and without some regard for 
consequences that could have the potential to 
do great harm. Therefore, the denial of a high 
school diploma because a student faMs to 
reach the cutoff point of an examination not 
only labels the person as a failure but may 
eliminate many job opportunities for which 
the poor test performance is irrelevant 
Holding back a student: because of poor test 
performance, or on any other basis, must be 
judged in terms of the likely consequences. 

Educational issues 

Given this issue, I am concerned with fair- 
ness, validity and the relationship between 
increased graduation requirements and 
academic performance. 

in judging the fairness of minimum com- 
petency testing to assess the standard of per- 
formance for graduation requirements^ the 
match between the test and what is actually 
taught is of particular importance. It is my 
belief that fairness requires that a school's 
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curriculum and instruction be matched in 
some way v/ith whatever is later measured by 
the test The test would be unfair if it 
measured what the school never taught. 

McClung [9*10] suggested that tradi- 
tional notions of content vaiidit/ should be 
expanded to include the concept of "instruc- 
tional validity". In other words, the validity 
of a minimum competency test should be 
established in part by examining the relation- 
ship between test items and instruction 
actually received by students. If a test 
measures outcomes never taught in school, it 
may violate substantial due process because 
the school rather than the students is at fault 
and the students are being punished without 
being personally guilty. 



Curriculum validity Is a measure of how 
well test items represent the objectives of 
curriculum to which the test-takers have been 
exposed. Even if the curriculum objectives 
for the school correspond with the compe- 
tency test objectives, there must be some 
measure of whether or not the school 
district's stated objectives were translated into 
topics actually taught in the district's class- 
rooms. If test items do not reflect actual 
content of instruction, then the competency 
test should not be used as a high school gradu- 
ation requirement for individual students. 

Ferratier and Helmick [11] report that: 

.... .there is no evidence that increasing 
graduation requirements affects student 
academic performance. In fact, there is 
some evidence to the contrary. State of 
Illinois and national data show that there 
is a weak or inconsistent relationship 
between graduation requirements and 
student achievement. This is a most dis- 
turbing finding since it is counter to most 
conventional wisdom. National ly [ 1 2] , 
there is evidence that school districts with 
higher achievement have fewer graduation 
requirements, while districts with lower 
achievement levels tend to use higher 
education requirements, probably in an 
effort to raise achievement. 



Pugh 

Assessing College Admissions 
Standards 

The concern for the increasing number of 
underprepared students now entering college 
and the precipitous increase in ithe numbers of 
students involved in remediation at the 
college level have prompted national studies 
and actions by individual states to address the 
probjem. Some states have enacted fcjislati ion 
or adopted state policies requiring institutions 
and state agencies to raise admission 
standards. 

In the last five years, six surveys of admis- 
sion standards at public colleges and universi- 
ties have been pubjished [13, j4,15,16,1 7, 
IS.] Four of the surveys are national in 
scope; two are for states in the South or West. 

The most recent ^natlon^d survey was con- 
ducted by the College Board [18] in 1986; 
This survey revealed that twenty-six states 
have no statewide admission standards and 
^? n ^"f9 ar _^J^J e _PP r ^ ( ' having statewide 
minimum admission standards^ In thirteen of 
these states the institutions of higher educa- 
tion are not allowed to exceed state require- 
ments, while ^ eleven^ states allow individual 
institutions the authority to impose more 
stringent admission standards; 

What kinds ^ new statewide admission 
standards are being used? In 1984-85, nine 
states required only that in-state students 
entering public colleges and [universities have a 
high school diploma or hs equivalent; Of 
these nme states, seven jldaho^ Kansas, 
Louisiana, Montana, Nebraska, North Dakota, 
and Wyoming) did not ^ allow Individual 
institutions to impose additional require- 
ments, making these state systems, in effect, 
open admission systems^ Two states (Ken- 
tucky and Ohio) permit thejr institutions the 
opportunity to impose stricter standards. 

tight states require entering freshmen to 
present satisfactory scores on college 
admission tests, test scores are the only 
requirement for admission to Mississippi's 
public university system. Applicants to 
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public universities^ and colleges in Florida 
mast present a mjnimam hi^i school GPA as 
well as satisfactory test scores; while students 
in Arizona 1 Georgia, Qjdaf^n^iL ® re 8 on f 
So^ti^ Dakota, and West Virginia can Be 
admitted __on the basis of tiieir GPA's or 
ano t her c ri ter |<>n, such as class rank or pre- 
dicted performance, if their test scores are 
below the minimum required. 

In addition to Mississippi, seven states use 
only one criterion for admission. Tfi^sing[e 
criterion used by Jhese states are: minimum 
GPA (Nevada and Maryland), class rank 
(lowa) y liij^h school coarse work requirements 
(New jersey and Wfcconsin), and a sliding 
scale model (Massachusetts and California). 

Current regular admission standards for 
in-state stu den ts to be a member institution 
of the University System of Georgia are a 
minimum GPA on academic work of 1.8 or a 
250 verbal SAT score or a 280 quantitative 
SAT score. Entering freshmen scoring below 
330 on the verbal S AT or bejow 330 on til le 
qualitative SAT will be given the ^ Regents 

Basic Skills examination to determine 

whether developmental studies shall be 
required prior to entry into regujar college- 
credit^ courses^ Spec|fic institutions of the 
system may inquire higher levels of perfor- 
mance requirements and/or additional testing; 

The University System ofJSeorgia is one 
of eleven statewide systems that have 
established statewide minimum standards 
with institutional discretion. Others are 
Arizona, Florida, Kentucky, Maryland, 
Massachusetts, Mississippi, K:w jersey, Ohio, 
West Virginia ^nd Wisconsin. The type 
of statewide admission standard in use in the 
State of Georgia as of J 984-1 985 was a 
minimum GPA or test scores. Only five other 
states beside Georgia use this type of state- 
wide admission standard; they are Arizona, 
Oklahoma, Oregon, South Dakota, and West 
Virginia. 



In light of extensive discussion about the 
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inadequate academic preparation of students 
entering college today, it is interesting to note 
that only five states (California, Florida, 
Massachusetts, New j ersey , and Wisconsin ) 
required a prescribed pattern uof high school 
coursework as an entrance requirement for 
their public colleges ^nd universitites In 
198435. The Univereity System of Georgia 
will join these ranks in the fal^of 1988. Four 
more states Jjjlinois, Kansas, Ohio, and 
Montana) recommended |_a minimum curric- 
ulum for entering freshmen. Twelve states, 
however, are moving to either recommend or 
require the adoption of high school curricu- 
lum standards. 

What are th e Semis in th e new ad m issi on 
standards or policies? During the period 
1 982-1 985^sixteen stares have enacted or are 
proposing more stringent statewide admission 
standards. Ten states reported no antici- 
pated changes in their admission standards. 
Two of these states {Kansas and North 
Dakota) chose to retain their open admission 
policies after careful study. 

in the s|xteen states making changes, the 
new or proposed standao-ds^ involve either 
imposing fourteen [States or strengthening two 

a __P r ^r'^^_P? tte J' , l_?^_ A'8h school 
coursework for entering freshmen. Two of 
these states also recently raised the minimum 
test scores (Florida) »id minimum grade 
point average (Oregon) required for 
admission, and two states; (Arizona and 
Idaho) will require students to complete their 
prescribed high school courses with a 
minimum GPA of 2.0; 

Statewide admission jnodels most fre- 
quentJy incorporate three major criteria: high 
school rank, h|gh school grade point average, 
?P5L a A te ?ikrdized test score. J_ n .addition 
these criteria can be a«d to develop two 
additional methods for admission; ^they are 
the sliding scale and predicted performance 
models. Breland^[l9] developed five state- 
wide admissions models and he assessed their 
impacts upon three racial groups. 
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Table 1 

SUMMARY TABLE FOR FIVE MODELS 
OF STATEWIDE ADMISSIONS POL 'CIES 



Perc«Mtage Eligible by Group 1 Differential Impact 

Models/Minimum Blacks Hispanics Whites Blacks Hispanics 



1. Single Index 





Rank in top 2/5 


56 


64 


71 


12 


7 




GPA - 2.75 


52 


65 


72 


20 


7 




SAT - 800 


27 


43 


73 


48 


30 


2. 


Multiple Index 














top 2/5 and SAT 500 


56 


65 


72 


10 


7 




Top 2/5 and SAT 600 


46 


50 


71 


25 


12 




GPA - 2.50 and SAT 700 


37 


55 


77 


40 


22 




Top 3/5 and SAT 800 


27 


43 


72 


40 


29 




GPA 


26 


42 


72 


40 


30 


3. 


Either-Or Model 














Top 2/5 or SAT 1100 


60 


67 


74 


15 


7 




Top 2/5 or SAT 1000 


60 


66 


76 


16 


8 




GPA 3.0 or SAT 900 


43 


61 


73 


30 


12 




Top 1/5 or SAT 800 


45 


57 


75 


30 


22 


4. 


Sliding Scale Mode|2 


45 


59 


75 


29 


16 


5. 


Predicted Performance^ 














Sample Inst. A 


40 


60 


72 


32 


12 




Sample Inst. B 


37 


58 


72 


35 


14 



Source: Excerpted from Hunter M. Ereland, An Examination of State University and College 

Admissions Policies Research Report 85-3. Princeton, N.J. ; Educational Testing $;rv!ce, 
1985. 
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Notes: 

1; Tlie comparisons in the table reflect 
situations where about three-fourths of 
white students are eligible, which is a 
common level of selectivity among public 
universities and colleges. 

2. Sliding scales typically emphasize high 
school rank or GPA at the higher ability 
levels! with either no test score require- 
ment or with a relative low test score 
minimum at the higher ability levels. Five 
different sliding scales were examined in 
the Breland study. The sliding scale pre- 
sented here represents the same level of 
selectivity as other examples in their 
table. In this sliding scaJe, all students in 
the upper tenth of their high school class 
mark are eligible; There is no test score 
requirement for stu lents at this level or 
rank. Students in Lhe second tenth are 
required to have an SAT combined scores 
(verbal plus math) of at least 800; 
students in the second fifth, 700; students 
in the third fifth, 900; students in the 
fourth fifth, 1100; and students in the iast 
fifth, 1300. 

3. Predicted performance models use a 
weighted combination of the high school 
record and test scores to predict college 
freshman GPA* For the analyses reported 
here, weighted combinations of high 
school GPA and SAT combined scores 
(verbal plus math) were used. Students 
predicted to attain freshman i CPA's of at 
least 2.60 were considered eligible. Teh 
sample institutions, all state universities 
and representing a range of activities, were 
examined in the Breland study. Sample 
institutions A and B presented here 
represent the same level of selectivity as 
other examples in this table. 

Conclusion 

In conclusion, this development of new 
standards for high school graduation and 



col lege admissions are undergoing rapid 
change. These changes will have differential 
impacts upon secondary and postsecondary 
institutions, students and educational pro- 
grams. It is not clear whether or not these 
changes in standards will improve the edu- 
cation quality of our students. 
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NEEDS ASSESSMENT 
IN VOCATIONAL TECHNICAL EDUCATION 

Kenneth R. Alien 
Georgia State University 



Needs assessment arid documentation 
have become a prevalent practice for voca- 
tional technical schools and programs in re- 
cent years. Since the 1976 Amendments to 
the Vocational Education Act of 1963, there 
has been increased emphasis on program 
accountability and heeds assessments have 
been utilized in the data gathering process for 
numerous types of evaluations. 

This discussion will focus on current 
heeds assessment issues in the State of Geor- 
gia in vocational technical education. The 
discussion will first focus on background in- 
formation so that the needs assessment pro- 
cess may be placed in perspective. 

Purposes of Post-Secondary Vocational Tech- 
nics! Education 

A primary objective of technical and 
occupational education in Georgia is to pro- 
mote economic development within the 
State. This is accomplished basically in three 
ways: 

1. By providing a basic work force 
which is technically competent 

2. By conducting specialized start-up 
training for new and expanding 
industries. 

3. By conducting short term training 
in almost any occupational area on 
an as heeded basis. 

A second objective is to serve as a link 
between schooling and work. That is, to pro- 
vide the post-high school age person with the 
opportunity for training in the occupational 
area of their choice. 



A third objective is to provide an op- 
portunity for continuing education opportu- 
nities of an occupational nature for the over- 
all community served by the school. 



Issues in Technical Education 

There are several big issues which are 
affecting the Vocational Technical Education 
System in Georgia at the present time. These 
are: 

Creation of a State Board of Postse- 
condary Vocational Education — Oh jujyj* 
1 985, Governor Joe Frank Harris appointed a 
State Board specifically to oversee the Post- 
secondary Technical Schools in Georgia. The 
transit ion from State School Board control to 
Postsecondary Board fs a development of 
superordinate importance to any other issue 
in the technical education community at the 
present time. This hew board, which is com- 
prised almost entirely of business and indus- 
trial leaders, is especially conscious of the 
need to plan arid evaluate decisions based on 
accurate heeds assessment information. 

Associate Degree Programs — The con- 
version of six schools in the system to Asso- 
ciate Degree status with the attendant move 
from the Commission on Occupational Edu- 
cation Institutions to the Commission on 
Colleges of SACS is a continuing area of con- 
cern to many in the Vo-Tech System as well 
as the University System. 



The final. effectiveness of these schools 
as college level institutions is an area of needs 
assessment that must be addressed. 

High Technology — The potential im- 
pact of technology on employment, and its 
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impact on occupational preparation programs 

is still only P^L a l[y_? n ^f!^^9di_^^l e _ t h e 
schools have conducted |_a ffezt deal of sfcidy 
on_analysis of this problem, a continuing ef- 
fort combining the resources of business and 
Industry w L*h_ _those_ _of _ _ed u ca t i on_ _ w il }_ _be 
needed to keep programs responsive to needs. 

Sources of Needs Assessment Data 

We at Georgia State University have 
b_ eeri _ ■ ll ^ 0 It e ^ JD collection of needs as- 
sessment date for use by state jevel decision 
makers for the past^ several [ months. Data of 
this type comes from a variety of sources. 
Key pieces of information which facilitate the 
needs assessment and subsequent decision 
making process in technical education are: 

State anH Federal Department of tabor 
and Bureau of tab or^ Statistics Data — This 
information is the most current source of 
labor supply and demand information. 

Census Bureau The Census Bureau 
provides demographics on population data by 
age, educational jevej, and current occupation 
which can be_ connected to sub state and 
|ooalized information, it is commonly used to 
generate a trend analysis of a specific geo- 
graphical location. 

Area Planning and Development 

Commission Data - The 18 APDC's jn the 
^tate of Georgia generate »i enormous a- 
mount of infoirnaticm _each year on the 
demographu: and economic trends jn their 
area. This data is of enormous importance in 
planning for technical programs. The APDC's 
are extremely important in moving from state 
level to more localized projections. 

teader^|p__Fonnns_ — Perhaps Se 
single most important data collection^ tool jn 
the needs assessment process is the gathering 
of first hand information from local employ- 
ers and community leaders. Approximately 
every two years a series of "teadership^ For- 
ums" is conducted by the Postsecondary 
Board in concert with the local schools and 
the Area Planning and Development 



Commission. In these meetings the local 
communities have the opportunity to offer 
direct input into the decision making process. 

Survey Data from t oca! Schools — Re- 
cently each technical school in Georgia has 
been called [ upon by the State Board to 
conduct a local community needs assessment. 
Should thisjprogram be successful it will pro- 
vide the primary data collection methodology 
in the future. 

Other Sources — Data from several 
other ^ sources is ajso Important, such zs the 
Department of industry and Trade Summary 
of ne^planf openings and closings as well as 
their manufacturers and in du stry directory. 
The Management information System of the 
Georgia Department of Education provides 
data on program enrollment and placement 
status. 

A somewhat obvious source of data 
J?_ often jwerjqoked for determining ^m- 
ployment trends is the "Help Wanted'' section 
of tiie classified ads of local newspapers. 

Purposes of Needs Assessment 

The purpose of a needs assessment is to 

determine what ought to be done based on 
current facts or trends compared to a set of 
predetermined expectations or criteria. This 
process is often called discrepancy analysis. 
Needs assessments may be conducted in edu- 
cational programs in a number of different 
functional areas. In the technical education 
system there are three primary areas of needs 
assessment. 

1 . Product evaluation — Data is collected 
9Jl_^ e _ _°*?tputs or products of voca- 
tional education, i.e. the students. This 
needs assessment process tries to 
determine what ought to be done rela- 
tive to the following questions: 
Are students enrolling in our programs? 
Are students successfully completing 
our programs? 

Are students finding employment after 
completion? 
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Are students considered successful in 
their employment over the long term? 

2. Employment Supply and Demand — 
Needs assessments in this area seek to 
answer the questions for what occupa- 
tional areas should we train students 
and how many should be trained in 
each area? These questions while seem- 
ingly simple are the most difficult to 
successfully answer. The final decision 
on programs to initiate, expand, or 
phase out, impacts on funds, space, 
equipment, and staff in a profound 
way. Successful program planning of 
this type requires a detailed analysis of 
a complex set of demographic and eco- 
nomic trends which sometimes resem- 
ble attempts to predict the stock mar- 
ket. 

3. Curriculum Development — Needs as- 
sessment in this area simply seeks to 
answer the question "What ought we to 
teach?" in a specific program. Maxi- 
mum currency and relevance to the real 
world of work should always be the 
goal of curriculum development in 
technical education. To that end, tech- 
nical educators routinely utilize con- 
tent experts from the business and 
industrial community to review curri- 
culum at all levels, even down to the 
syllabus level. These advisory groups 
are instrumental in setting instructional 
objectives for courses and programs and 
are the primary source of needs assess- 
ment data. 



Results of Needs Assessment 

Current heeds assessments conducted 
by Georgia State University in the past several 
months have indicated several areas of con- 
cern which must be addressed by the techni- 
cal education system. These are presented 
below in no particular order of priority. 



SI 



Image of VcnTech Education — The 
State System must address the fact that occu- 
pational education does hot enjoy a particu- 
larly favored position in the eyes of many 
potential students, particularly those in the 
18-25 age range. 

Economic Development — In a some- 
what paradoxical point to the image problem 
cited above, it is evident that business, 
community, and industrial leaders across 
Georgia have a strong perception that Vo- 
Tech education is a critical factor in the eco- 
nomic well-being of the state. 

the Work Ethic/Attitudes — . In a 
series of 23 Leadership Forums conducted in 
every planning district in Georgia, the GSU 
research team was told in every meeting that 
Vo-Tech programs and general education 
programs should do all in their power to stress 
work related attitudes including the overall 
willingness to work at gainful employment, as 
well as basic procedures such as dress, 
punctuality, and relations With others. 
Customer relations Was another key area of 
emphasis by employers. 

Job Availability — There are numerous 
well paying jobs in Georgia for people willing 
to get training and work up the career ladder. 
There are^ however ^ few takers in many tradi- 
tional trade program areas such as metals, 
mechanics, heating and air conditioning, and 
construction trades. Whether this is the result 
of the economy, the image problem of Vo- 
Tech Education, or work ethic deterioration 
is unclear, it is, however, a dilemma that must 
be solved. 

Business-Education Partnerships — As 
perhaps at no other time in the past, there is a 
heed for business and education to work to- 
gether to develop, deliver, and assess educa- 
tional programs. The Quality Basic Education 
Act, economic development trends, and the 
impact of technology have all contributed to 
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this need, and in general it is educators who 
have not responded quickly enough to form 
more cooperative arrangements. 



Trends in Needs Assessment 

Briefly, the following trends appear to 
be at work which will require a more thor- 
ough needs assessment process. 

Educational programs will Be evaluated, 
as are industries, oh the basis of the quality of 
their product 



Changes in technology wiH continue Jo 
require thai the curriculum be modified as the 
world of work changes. Continuous feedback 
from the business and industrial community 
to the educational community will be needed. 
What engineers cajl a "Closed- Loop" instruc- 
tional system will be required. 

As education continues to be a prime 
factor in economic development, business and 
education must move closer and closer into 
partnership. 
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THE NEW CRITERIA FOR ACCREDITATION 

James T. Rogers 
Commission on Colleges 
Southern Association of Colleges & Schools 



Good Evening. I bring you greetings 
from the Commission on Col leges and [the 
entire SACS staff ^ Lei : me first say that I 
^j^Pi 11 ? _* e 0 PP°^r!ty_i > f_?ddre»ing 
conference. As a former college president in 
this state for 15 years, I know full well that 
the institutions of higher education in 
Georgia-both public and private^ junior and 
senior-are some ^of the finest in to is country. 
I also know from my presidential experience, 
and from my new prespective at the 
Commission on Colleges, that some very 
exciting developments are taking place right 
here in Georgia. 

I am particularly grateful to have the 
chance of talking with you about SACS 9 new 
"Criteria for Accreditation." j sincerely 
be iieve that th is new development p laces all 
of us, not only on the cutting edge of Amer- 
ican higher education, but also at the fore- 
front of regional accreditation. 

In recent years, regional accreditation has 

e AP B l'?n?_ e ^ JP5_ r ® as L l !8 _ P r ? s f are _*9 ly?!^ 
institutions more accountable for their 

educational process. It has also been under 
attack by its critics, and especially during the 
last decade, has been criticized [both from 
^i^lL 1 ! i iri ^f r _ om _ Y?^9 u Jif °L fie^yrel iahce 
2H_ J ra Afr'_ on ?l evaluatjon me^re^measures 
ffiat have focused almost exclusively on 

processes and resources^ Much of the 

criticism has been legitimate, especially as 
new kinds of ^ijist|tutions h^e evolved and as 
the more established colleges and universities 
have developed jiew kinds of programs and 
innovative delivery systems. 

The SACS leadersh ip recogn ized some 
YA 31 ^. a 8P * at __ J&tne _ h a d c ome to 
reexamine its standards and procedures in 
order to determine whether new approaches 
were called for. in doing so, the Commission 



also understood that the old standards, which 
had focused on resources and processes, had 
^_?9^_ us J n 3°°A ^® a 4 fpr many years _and 
that it would not Be in the best jnterests of 
our membership to completely abandon the 
old and go with something entirely new. 

With this in mind, it js important to spend 
just a few minutes placing the new "Criteria" 
in an historical perspective. 

In 1980, the Commission j>n Colleges 
conducted a survey of its membership to 
detenu ine their perceptions abou it accredi- 
tation. The study included college adminis- 
trators and faculty, as well as local, state and 
federal agency personnel. Almost 2,000 
educators throughout the nation were 
surveyed. The results of this survey, which 
served as a kind of foundation for the project 
to revise the standards for accred[tation, 
indicated that basically the respondents were 
satisfied with^ the association P^^cdures 
which had remained relatively unchanged for 

over two decades. The respondents also 

strongjy endorsed tfie self-study process, but 
felt that accreditation standards rejied too 
heavily upon resources and should begin to 
place a stronger emphasis on the results of the 
educational process, namely, student learning. 

As a result of this very important survey, 
the leadership of the commission, with the 
endorsement of the membership, initiated the 
project to revise tfie standards. Throughout 
tfie effort, over ^6 people, working through 
almost a dozen committees, helped to give 
shape to the "Criteria." Six seminars were 
held throughout the southern region to dis- 
? uss J*?? 0111611 ** When the "Criteria" was 
presented to the membership in 1983, it was 
adopted in principle. However, concern was 
expressed by a significant percentage of 
the n rtbership about the new thrust on 
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outcomes assessment A number of people 
disagreed [with the philosophy of outcomes 
assessment and felt that too heavy a reliance 
9D J>?teomes_sTOssment would reduce the 
educational process to elements which could . 

only be quantifiably measured. While 

approving the outcomes assessment approach 
in pnnciple, other Individuals L felt that tfiis 
D e ^y JVPr°* c h would make accreditation too 
bureaucratic or too costly. Thus, in response 
to the criticism of Jhe mehiberehip, the 
chairman of the Commi^ion^ on Colleges 
withdrew the section on outcomes assessment 
and appointed a special committee to review 
this new thrust 

The review committee was charged with 
the responsibility of not only editing the 
"Criteria," but also preparing a new section 
whichjvould replace the original statement on 
outcomes assessment. 

Every member institution was given an 
°PP°rtunjty to jubmit ite concerns and 

suggestions to the committee, and the 

committee addressed these problems during 
the sp^ng and summer of 1 984. W particular 
concern to the review committee was that the 
new statement not be overly prescriptive, and 
that institutions be allowed a great deal of 

flexibility in developing individual 

approaches. , The committee also felt that 
there were too many negatives associated with 
the term "outcries assessment" and that in 
its place the terms "p[ann|ng and evaluation' 9 
should be used; There appeared to be little 
disagreement that tiiese terms more deafly 
describe what is needed to assess student 
achievement. The institutional research 
function was incorporated into this new 
section because committee members strongly 
felt thai institutional research was a critical 
element in any effective planning and 
evaluation process. With this in mind, the 
committee adopted the term "institutional 
effectiveness" for this section of the 
1 'Criteria", and incorporated planning evalua- 
tion and institutional research under this um- 
brella term. When the new section i__orj i insti- 
tutional effectiveness was presented to the 
eo|leg* Delegate Assembly in December 
1984, it was adopted overwhelmingly. 



Thus, with the adoption of the "Criteria 
for Accreditation" jn [1984, this document 
replaced the "Standards of the College 
Delegate Assembly." 

Before we examine Section iji of the 
"Criteria" more closely, we should, perhaps, 
ta|k ajittie about how the "Criteria" differs 
from the "Standards." There were, for exam- 
ple, e|even standards. The "Criteria" Jias six 
sections, essentially addressing the same edu- 
cational functions, but organized in a more 
consistent fehion. All the former sections of 
the old "Standards" dealing w|th the educa- 
tional program can now be found in one sec- 
tion in the "Criteria." Likewise, all purely 
administrative processes and educational 
support services are together in one section. 

Section I, Principles and Philosophy of 
Accreditation, includes the 14 conditions of 
eligibility for candidacy and the two 
cqnd itions of eligibility for membersh ip. 
These conditions, by the way, will be 
reviewed by a special committee this spring 
and recommended changes will be brough t 
before tiie College Delegate Assembly next 
December. 

Section^ II, Institutional I Purpose, requires 

an institution to examine its purpose. 

tankage in this section also requires that the 
planning and evaluation processes be designed 
to demonstrate that the institution's purpose 
and role are being fulfilled. 

Section l|l, Institutional Effectiveness, 
Includes subsections on planning and 
eval u at i on an d in sti tu t i onal researc h^_ tl n der 
the old "Standards," it was assumed that If an 
institution had certain resources and used 
certain processes, effective education would 
occur. This particular section states that 
while resources and processes are important, 
tiie evaluation of the results of education and 
plans for the Improvement of the institution 
are equally important 

Section IV, Educational Program, requires 
that tiie principal focus of an institution be 
the education of its students. It also requires 
that all aspects of the educational program be 
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clearly related to the purposes of the 
institution. included in Section IV are 
subsections on the undergraduate program! 
graduate program, continuing education, and 
faculty. 



Section V, Educational Support Services, 
includes the library! instructional support, 
computer services, and student development 
services. 



And finally, Section VI, Administrative 
Processes, includes organization and 
administration, financial resources, physical 
resources, and externally funded grants and 
contracts. 

Throughput all of these sections there is 
language addressing planning and evaluation 
processes, 

The committees which worked on the 
"Criteria" made a concerted effort to clarify 
the language in the document and to make 
the new language less ambiguous than it had 
been in the "Standards." Under the "Stan- 
dards/' for example, "musts" and "shoujds" 
were never clearly understood by the institu- 
tions being reviewed or by the visiting com- 
mittees. In the "Criteria," a "must" statement 
represents something that is clearly a re- 
quirement "Should" statements, on the other 
hand, indicate something that is advised. 

Finally, and most importantly, the 
"Criteria" differs from the "Standards" in 
that Section III, "institutional Effectiveness", 
represents an entirely new thrust in the 
accreditation process. 

At this point, it is important to point out 
that Section III of the "Criteria" has fewer 
"must" statements than any other of the 
sections. The five requirements established in 
the section of institutional effectiveness are 
that: 

1. Institutions must establish adequate 
procedures for planning and evaluation. 

2. Institutions must define their expected 
educational results and describe how the 



achievement of these results will be 
ascertained. 

3. institutions with research and public 
service missions must develop and 
implement appropriate procedures for 
evaluating their effectiveness in these 
areas. 

4. institutions must engage in continuing 
study, analysis and appraisal of their 
purposes, policies, procedures and 
programs. 

5. Institutions regularly must evaluate the 
institutional research function. 

Oil the other hand, there are a number of 
statements in this section which suggest what 
an effective planning and evaluation process 
should include: 



1. Broad-based involvement of faculty and 
administration; 

2. The establishment of a clearly defined 
purpose appropriate to collegiate 
education; 

3. The formulation of educational goals 
consistent with the institution's purpose; 

4. The development of procedures for 
evaluating the extent to which these 
educational goals are being achieved; and 

5. The use of the results of these evaluations 
to improve institutional effectiveness. 

The very last paragraph of the subsection 
on planning and evaluation, however, very 
clearly indicates that "the appropriateness of 
any evaluation procedure depends upon the 
nature of the institution and institution's 
goals for instruction, research and public 
service." Further, the section very clearly 
states that "the Commission on Colleges 
prescribes no set of procedures for use by an 
institution and recognizes that an effective 
program to evaluate institutional effectiveness 
will usually require the use of a variety of 
procedures." 
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Thus, the section on institutional 
effectiveness, while requiring institutions to 
adopt planning and evaluation processes, 
aliows institutions to develop these processes 
in accordance with their own goals- v<th out 
restriction and without specific data- 
collection methods. While it provides guide- 
lines for improvement, the section imposes no 
specific performance standards nor does it 
mandate any particular approach to planning 
and evaluation, institutional effectiveness 
also encourages institutions to continue the 
momentum that results from the self-study 
process, in other words, it emphasizes the 
need for an institution to Invest in the 
ongoing capacity to study itself. 

Historically the self-study process, while it 
has been a very important part of institutional 
evaluation, has been for many institutions an 
isolated event occurring only once every 
decade, it has not, until the adoption of the 
"Criteria for Accreditation," and even more 
particularly the adoption of Section III of the 
"Criteria/ 9 required an institution to have an 
ongoing capacity to study itself. The new 
emphasis on institutional effectiveness 
encourages the administrative leadership of an 
institution to use an ongoing planning and 
evaluation process as the basis for major 
decision making activities at the institution, 
thereby providing clearer options for policy- 
making and problem-solving. 

Perhaps most important of all, however, is 
that the new emphasis on institutional 
effectiveness will result in a more integrated 
campus-wide planning process. In far too 
many institutions this effort has been frag- 
mented. It also encourages the use of a 
common data base using accessible and 
reliable data which can, by producing 
standardized information, assist management 
in functioning more effectively. 

From another perspective, the new 
emphasis on institutional effectiveness gives 
the visiting committee a more appropriate 
mechanism for evaluating all types of learning 
experiences, thereby giving institutions a 
greater freedom to explore new teaching 
methodologies to meet the special needs of a 
new generation of learners. 



Virile I have mentioned some of the 
positive effects that the "Criteria" can have 
on member institutions, I must, at the same 
time, acknowledge that the "Criteria" does 
not provide all the answers and, indeed, even 
raises for all of us some very difficult 
questions: 

To name only a few: 

1. What guidelines, for example, will visiting 
committees use to determine the 
"adequacy" of an institution's planning 
and evaluation efforts? 

2. What kinds of "specific" skills do 
members of visiting committees need to 
effectively evaluate these efforts? 

3. How can the Commission on Colleges best 
train its future visiting committee 
members to adequately assess the 
effectiveness of an institution? 

4. What resources can be made available to 
member institutions to help them develop 
effective and cost-efficient processes? 
How can the Commission best serve its 
constituency in this area? 

We are working to resolve these and other 
questions. 

Ladies and gentlemen, what we have in 
our new "Criteria" is an opportunity to ex- 
plore r < d learn together-for an accrediting 
agency and its constituent members to en- 
hance the credibility and accountability of 
both. The new section on "institutional 
Effectiveness" is a very modest, first step. We 
continue in our conviction that outcomes 
assessment must have its place in the legiti- 
mate concerns of the accrediting process. We 
are steadfast in our conviction that it is both 
reasonable and necessary to require that an 
accredited institution be able to describe what 
it is trying to achieve, how it measures the ex- 
tent of that achievement, and that it is achiev- 
ing its objectives to a reasonable degree. 

Indeed, we are equally convinced that if 
we fail in this endeavor, the future of private, 
voluntary, regional accreditation may be in 
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question. And we do hot have the luxury of 
debating the issue for very long. Either we do 
it, collectively through accrediting standards, 
and individually through an institutional 
commitment to be truly accountable, or it is 
likely that other agencies will do it for us-and 
to us. That's not running scared. That's 
facing reality. 

Quality judgments about higher education 
are presently in the hands of the so— called 
"triads-state government* the federal 
government, and voluntary accreditation- 
each with its legitimate role. The possibility- 
through abandonment, default, or abro- 
gation-of losing the self-regulatory element of 
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that triad and leaving quality assessment and 
control solely in the hands of governmental 
agencies, is not very palatable to most folks in 
higher education. The appropriate imple- 
mentation of procedures for assessment 
of outcomes, under whatever rubric, is crucial 
to the retention of the self-regulatory element 
of the triad. 

Charles McClain, Northeast Missouri State 
University: 

"I think the public wants assurance today 
that a college degree has some integrity- 
thai what it represents isn't being left to 
chance." 
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THL IMPLICATIONS OF NEW SACS CRITERIA FOR INSTITUTJONAL RESEARCH 

Larry G. Jones 
Institutional Research and Planning 
The University of Georgia 



Accreditation, or more precisely the self- 
study process upon which accreditation is 
based, has always had significant implications 
for the function and process of institutional 
research. Self-study, after all, is institutional 
research and institutional research is self- 
study. 

Recognizing the role of institutional 
research _ in the accreditation/self-study 
process, if not the more pervasive evaluation 
of the institution's self-proclaimed mission, 
earlier standards of the Southern Association 
of Colleges and Schools (SACS) carried the 
following statement about the necessity of 
institutional research and planning in "sound" 
administration and the importance of "con- 
tinuing study, analysis, arid appraisal of 
[institutional] purposes, processes, arid pro- 
grams." The results of institutional research 
were to be evident in the "major decisions 
and projections" shaping the institution's 
operation and its "academic and administra- 
tive functions." In short, the SACS- 
accredited institution was expected to do 
institutional research. 

Institutional Research and Planning 

Institutional research and planning are 
necessary functions of sound adminis- 
tration. Organization of these responsibil- 
ities will depend upon the size arid com- 
plexity of the institution; however, all 
institutions should engage in continuing 
study, analvsis, and appraisal of their 
purposes, processes, and programs. The 
results of these studies should be evident 
in the consideration given to major 
decisions arid projections which shape the 
future course of the institution's oper- 
ation. Both academic arid administrative 
functions should show evidence of 
continuing and systematic critical 
analysis. Responsibility for direction and 
coordination of these studies should be 



explicitly assigned to one or more staff 
offices which should be provided the 
necessary resources and access to infor- 
mation sources to discharge these duties 
effectively. ("Standards, of the College 
Delegate Assembly," Commission on 
Colleges, Southern Association of Colleges 
and Schools, Atlanta, 1977, p. 8) 

In fact, the mention of institutional 
research in the SACS Organization and 
Administration Standard probably gave 
considerable impetus to the creation of 
institutional research and planning offices and 
positions at colleges arid universities in the 
Southern Association states, it did so at the 
University of Georgia. The new criteria 
promise to be an even greater impetus for 
such offices. 

The prominence SACS accorded to 
institutional research in the accrediting 
process arid in the evaluation of institutional 
vitality helps explain why the South has 
played so significant a role in the devel- 
opment of institutional research as a practice 
arid profession. It would be ironic if the 
SACS "Standards" actually had been as 
instrumental in establishing institutional 
research in southern institutions as suspected, 
for a small but distinguished group of 
southern institutional researchers argues that 
the new SACS "Criteria" is the inevitable 
response to the questions raised arid answers 
given by institutional researchers to the issues 
fawirig higher education. T^is group of 
colleagues woulo contend thai my paper 
should be titled "The Implications of Institu- 
tional Research For New SACS Criteria," 
rather than the "Implications 6f_the New 
SACS Criteria for Institutional Research." 
Regardless of which side of that argument one 
chooses to take, the implications for institu- 
tional research in the new SACS "Criteria" 
are significant 
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An increasingly important function of 
institutional research is to monitor internal 
and external pressures on higher education. 
One can assume that the College Bejegzte 
Assembly was responding to an environmental 
assessment when it set cut to re vise the SACS 
"Standards." There is a very j:jear public 
concern for quality and accountability in all 

^ ^^^^^ AQ 0 ^^?^ 1 ? 1 ! rein'forccd by 
institutional research (if not discovered^ by it 
in the first place). The SACS response to 
internal and external [ pressures, and the 
resultin^ M Criteria^' underscore the depend- 
ency relationship between institutional 

research and accreditation. The creation of 
the new criteria established a different 
emphasis in the accreditation process, 
provided a new focus for institutional self- 
study, and made a dramatic statement about 
what was important in higher education In 
the words of the "Criteria": 

Jfi^ jm 3 ''^ _°f education provided_ 

■n^nrtber institutions is the primary con- 
sumption the decision to confer or re- 
affirm accreditation^ The evaluation 
of educational quaHty is a difficult tok 
requiring ^ careful^ analysis and professional 
judgment Traditional ly, accreditation 
has focused attention almost exclusively 

upon institutional resources 3*}4 

processes. It : hzs usually been assumed 
that, if an institution has certain resources 
and uses certain processes, effective 
fducati on _ ^UL^ccur. A com prehensive 
approach to accreditation, L however, takes 

!!lt9._???P u D^ not J? ll 'y_ t ' 1 ? resources ar L < ' 
processes of education (such as faculty 
and student qualifications, physical plant, 
fiscal resources and other elements 
addressed in the Criteria] but also the 
evaluation of the results of education and 
plans for the improvement of the 
institution's programs. 

The |evel of institutional quality depends 
not only on an institotiqn's educational 
processes and resources but also on the 
institution's successful use of those 
processes and resources to achieve estab- 
lished goals, institutions have an obliga- 
tion to all constituents to evaluate effec- 



tiveness and to use the results jn abroad- 
based, continu ous planning and evaluation 
process; j"Crit2ria for Accreditation," 
Commission on Colleges, Southern 
Association of Colleges and Schools, 
Atlanta, 1984, p. 9) 

While the implications for institutional 
research in that statement are clearly more 
implicit than explicit, they become much 
clearer in the speciffc reference to Institu- 
tional research in the "Criteria": 

3.2 Institutional Research 

Because institutional research can provide 
significant information on all phases of a 
college or university program, it is an 
essential element in planning and evalu- 
ating the institution's success in carrying 
out its purpose. The nature of the institu- 
tional research function depends on the 
size and compjexityj>f the institution and 
may vary from a part-time operation to an 
office staffed by several persons, ^ How- 
ever, all institutions must engage in 
continuous study, analysis and appraisal 
of their purposes, policies, procedures and 
programs; Institutions should assign ad- 
ministrative responsibility for canning out 
institutional research. Institutional re- 
search should be allocated adequate re- 
sources, and those, responsible for jt 
should be given access to all relevant in- 
formation. Institutions regularly must eva- 
luate the institutional research function, 
(p. 10) 

Although the I R section in the ''Criteria" 

has n bt changed dramatical ly from the 

previously quoted statement found in the old 
"Standards/' the differences are still quite 
significant 

Fjrst of all, the institutional research 
statement has been placed in the Institutional 
Effectiveness section of the new "Criteria 0 
the focal point of the document. 

Second, the two "must" imperatives in 
the Institutional Research statement may be 
the most important of the new criteria. (All 
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"must" statements in the "Criteria" are to be 
interpreted as imperatives, whereas all 
"should" statements are advisory.) The first 
great commandment of accreditation under 
the new SACS "Criteria" is ". . . all institu- 
tions must engage in continuous study, 
analysis and appraisal of their purposes, 
policies, procedures and programs." The 
second commandment of SACS accreditation 
is like unto it: "The institutional research 
function must be regularly evaluated. " On 
these two statements hang all the process and 
procedure of institutional accreditation. 
Could there be any criteria more central to 
the en tire accredi ting process? Under the old 
"Standards," an institution was expected to 
do institutional research. Under the new 
"Criteria;'^ institutional research literally and 
fij^ratiyejy must be done to gain 
accreditation. 

As critical as institutional research is in 
accreditation, it's not an end in itself but 
rather a means to the end. By "Criteria" defi- 
nition, institutional research is "the contin- 
uous study, analysis and appraisal of institu- 
tional purposes^ ^ojicies, procedure and 
program s." Ad [hoc teleph one surveys of the 
"old buddy network," occasional campus 
questionnaires, and f acu Ity/admin istrative 
committee reports are not what the delegates 
had in mind when they wrote the IR 
criterion. Self-study has become by mandate 
a "continuous" process, not an exercise 
undertaken once every decade. 

The '•Criteria" says nothing about how 
die institutional research function is to be 
carried out, only that administrative respon- 
sibility for die activity should be assigned. 
There js no mandate for an office of institu- 
tional research or even for someone called an 
institutional researcher. By simple logic, 
however, anyone conducting institutional 
research is an institutional researcher. In light 
of what is expected of institutional research 
activity under the "Criteria," it js extremely 
difficult to imagine how the IR function can 
be carried out without a coordinator. 
Coordination does not necessarily mean 
centralization, but it is essential for the 
efficient and effective use of institutional 
resources. 



The new "Criteria" also refocuses the 
institutiona 1 research effort The old 
"Standards" directed the self-study effort 
toward quantitative and historical records. 
The emphasis was on attainment. The hew 
direction is qualitative, process-oriented, arid 
directed more toward accomplishment. 
Although the "Criteria," like the 
"Standards/' Jives relatively little specific 
direction to the ongoing IR function, there 
are some clues concerning priorities in the 
accreditation review. Two central themes in 
the new "Criteria" with considerable impli- 
cations for IR are planning and evaluation. 

institutional planning has enjoyed such a 
close relationship with institutional research 
that the two are typically linked in office and 
function titles. Although some assume that 
the two are separate functions, they really are 
not. Planning requires institutional research, 
and institutional research will inevitably lead 
to planning. 

Clearly, institutional research is a required 
part of the planning process directed by the 
"Criteria." The direct implication for institu- 
tional research is the imperative that ". . . the 
planning and evrJuation processes must be 
dt^gned to demonstrate that the institution's 
p! <^ose and role are being fulfilled." The 
"0, hsria" continues: 



Its acfdftioh to establishing procedures 
for sv^catitjj. the extent to which their 
ester tfjc^al £ sis are being acf ieved, 
jtfiyw ifcv4d ascertain periodically 
the in th* academic achievement 

of £-t*ir sraidBni*. Procedures used to 
evaii ! iS4j ^itruc^iglnal programs may 
incfuoa: v'°s evaluation of educational 
program; structured interviews with 
studv-nfe VrJ. grac"ja?es; changes in 
students' va^'es ?s measured by standard 
instruments or self-reported behavior 
patterns; pre- and post-testing of students] 
surveys of recent graduates; surveys of 
employers of graduates; student scores on 
standardized examinations or locally con- 
structed examinations; performance of 
graduates in graduate school; performance 
of graduates of professional programs on 
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licensure examinations; or, the placement 
of graduates of occupational programs in 
positions related to their fields of prepara- 
tion, (pp. 9-10) 

in addition to the required evaluation of 
the instructional mission, the new "Criteria" 
emphasizes . that institutions wish 

research or public service missions must 
develop and iMP^jnetit appropriate pro- 
cedures for evaluating their effectiveness in 
these areas.'^ And, to avoid any doubt about 
what constitutes appropriate evaluation pro- 
cedures, the document goes on to say: 

the appropriateness of any evaluation 
procedure depends upon Jhe nature of the 
institution and the institatjon's goals for 
instruction, research and^ public service. 
The Commissi on on Col|eges prescribes no 
set of procedures for use by an institution 
and recognizes th£t an effective j)royam 
to evaluate institutional effectiveness 
wiN usually require the use of a variety of 
procedures, (p. 10) 

In my opinion L the "effective program to 
evaluate institutional effectiveness" is 
identified as institutional research by the 
"Criteria." 

The lack of specifics j_and_ the emphasis on 
finding suitable institutional procedures in 
planning and evaluation are significant jn 
more than ^a pfiilosoph|cal^way. For many 
institutions, the |ack_ of specificity in the 
institutional effectiveness section^ be a 
Burden L not a Benefit The Timmoti ru\z of 
accreditation self-study has be: i tofci^^w the 
"book," snd a lack of directjon msv sirapsy 
frustrate the faculty and administration uSer 
than stimulate creative respo^s^H-My 
L nst L t _ l,t '9_ r ! s ^ hjave a significant bo^stiis up* 
?op_ r A'PA^_' ns t'^^ effort, *n:s 

therefore may lack the resources or jsxpertis-: 
to dedicate to the require^ IR Sfi. ;; t; To 
rescue, of course J w|ll j:qme M;ncic • *:v« ; :>'. 
forms of products to meet the tR_fa*/:f:L : n:«' 
for such institutions. That camyyi oz ?!f > ii\ 
but as a solution it does pose interesting r; 
for evaluating the institution;:* r;.jr*srch 



function and its role in the accreditation 
process. 

A final major implication for institutional 
research is the mandate for the ongoing 
evaluation of the institutional research func- 
tion^ The form and substance of such evalu- 
ation is not clear; there are no criteria for this 
evaluation either. Nonetheless, to keep in the 
spirit of the "Criteria," several points should 
be ! true about the evaluation of IR, JHrst, 
institctiona| research and researchers should 
be extended the same courtesy as other units 
on campus by having professional colleagues 
do the evaluation. "Evaluation [of institu- 
tional research j . A . is a difficult task re- 
quiring careful ^ndysis and professional 

judgment," to paraphrase a "Criteria" state- 
!5f Some common set of evaluation 
standards ^for the institutional research 
function should be devised to assist the 
institution preparing for the review and to 
help mostly unknowledgeable reviewers 
examine the institutional research activity. 
Because an institutional researcher will 
probably not^ be included on every SACS 
visitation team (there aren't many IR types to 
begin with, , and jfewer_ still on the SACS 
examination rosters), some effort wjll^have to 
be made to acquaint visitation team members 
with what IR does. Tojhisjihd, the Southern 
Association for Institutional Research has 
offered to provide SACS ^wjth whatever help 
it can in preparing visitation teams for this 
directed responsibility. Whether intended or 
not, the new SACS ^Criteria" has made 
Institutional research the p|votal process |n 
accreditation Jt may^erefore be pmdent 
for SACS Jo provide more direction for its 
effective practice ?eici evaluation, jrhe^clearer 
intent of the "CEferia M js to improve the 
qjtK'Jity of dec!S!or^f!jaking and educational/ 
institutional outeoritcs through fnstitutional 
re?^Tch. In fact, there is not a perfect 
cqnr lation between Me quality of institu- 
tional research and the quality of institutional 
pur - ng and decision-rc~ king; and the evalu- 
ation of the iR process cannot be judged 
comply «y on an evaluation of the an tic i- 
pss?«;,< or desired institution 1 outcomes. The 
t^cw SACS "Criteria" shodld improve the 
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relationship between IR and decision-making. 
If it does, the contribution it makes id the 
quality of higher education will indeed be 
noteworthy. 
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tinder the new M &i»rta/' it is possible 
for an institution to dp institutional research 
and not be accredited. It is riot possible, 
however, for an institution to be accredited 
without doing institutional research. 
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IMPLICATIONS OF SACS' NEW CRITERIA FOR 
DEVELOPMENTAL STUDIES 

Leroy Ervin and Louise M. Tomlinson 
Division of Developmental Studies 
the University of Georgia 



The question of implications of the 
new "Criteria" for bevejopmentai Studies 
raises some considerations about the processes 
involved in inipiem^nting those criteria in 
terms of the new standards for high school 
graduation and college admissions. These 
considerations indicate that impact upon 
developmental programs will be very gradual 
and that change in the structure or purpose of 
developmental programs, in response to the 
new criteria, may npt^_^^f^_j^enmtnw 
to ten years. The critical factors which will ul- 
timately affect developmental programs are 
teacher train jng, jegisjation, and the assess- 
ment and evaluation processes at the secon- 
dary and post-secondary Jevels, Each of these 
factors will be significant in determining if; 
when, or how the purpose, contr^t, or scope 
of developmental programs will .unge, and a 
- considerable amount of time will elapse be- 
fore any of these barometers will register new 
directions for post-secondary remedial pro- 
grams. 

Teacher Training 



Since the existence of Developmental 
Studies programs is contingent upon stan- 
dards of admission of our institutions of 
hjgher education, teacher training will; in 
part, be a determining factor in directions for 
post-secondary remedial programs. Teacher 
effectiveness will, to • a great extent, influence 
the success ratios of students who wiij_ at- 
tempt to meet the new "Criteria" for regular 
admissions to_ the_ university system. The 
minimum standards, four units of Engiish, 
three unite of math, three units of science, 
three units of social science, and two units of 
foreign [anguage, required for entry into the 
post-secondary system, will, no doubt, create 
a need for revised teacher training, programs. 



At this time we must consider some of 
the following questions about teacher person- 
nel. Is the supply there to fill the need? Are 
there too many teachers in the system who 
hold provisional certificates? Is there ade- 
quate quality control in teacher education at 
the college level? And, are we, therefore, lack- 
ing quality as well as quantity in our current 
teacher force? The answers to these questions 
suggest that, without some changes in teacher 
training, there will still be considerable num- 
bers of high school graduates in 1988, and 
beyond, who will fall short of the new stan- 
dards and will, therefore, seek provisional 
admission into Developmental Studies pro- 
grams. 

The rate at which teacher training pro- 
grams impact upon secondary education will 
determine the extent to which change will 
take place in Developmental Studies pro- 
grams. And, again, it will take time before 
such impact will be affected. 

Legislation 

Another factor that will, in part, be in- 
fluential in the direction that post-secondary 
remedial programs will take is legislation. 
Legislation is, essentially, the foundation for 
funding, if we consider the fact that the hew 
statewide standards encompass a number of 
locales that vary in socioeconomic character- 
istics, then we must take into account the fact 
that the success ratios of secondary schools in 
producing graduates who will meet the new 
"Criteria 1 ' will vary tremendously, from dis- 
trict to district, particularly at the outset 
Where the "Criteria" is most in conflict with 
the values and resources of the school 
community, there will likely be a greater 
elapse of time before the "Criteria" is wholly 
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implemented and manifest in the products of 
the schools. 

Unless state funding initiates and fol- 
lows some continuous formula based on need, 
to equalize the implem entation of the hew 
"Criteria/ 9 then there will be a lag in imple- 
mentation and achievement, By asking ques- 
tions like "what basis is there for creating 
equality?" and "how long will it take?", we 
must realize that, for some time to come 
there will still be a need for Developmental 
Studies programs. 

Assessment arid Evaluation 

Finally, the assessment and evaluation 
of the implementation and achievement of 
the new "Criteria'' at the secondary level will 
also be an influential factor in determining 
hew directions for post-secondary remedial 
programs. Whether pedagogical practices pro- 
vide adequate momentum toward the goals 
set by the College Board, how individual per- 
ceptions influence the mechanism, and the 
extent to which the hew criteria is met* are 
some of the considerations which will require 
thorough investigation. The difference bet- 
ween intent and effect has been recognized by 
Popkewitz, Tabachnick, and Wehlage (1982) 
as an important consideration in assessment 
and evaluation of reformed curricula and ad- 
ministrative procedures when introduced into 
the "real world" of schools. As a result of 
their investigation of a pabular reform 
program and its institutionaSizuion in several 
schools, these researchers formulated the fol- 
lowing questions. 

"How do specific social/cultural char- 
acteristics filter into the school to in- 
fluence instructional practices? What 
are the implications of teachers' diffe- 
rent and potentially conflicting percep- 
tions of their occupational role? Since 
professionals do have relative autono- 
my in establishing pedagogical prac- 
tices, how is this autonomy exercised 
to create, sustain, and renew occupa- 
tional ideologies? And, what are the 
roles played by state and local educa- 
tion agencies, teachers' associations and 



unions, and teacher preparation in- 
stitutes in establishing and legitimating 
the hew criteria?" (p. 180) 

These and a host of other more tradi- 
tional questions need to be answered, hot 
only by quantitative, but by qualitative in- 
vestigation which would include in-depth 
questionnaire and field study techniques. 
Assessment and evaluation will take time. 
Again, the direction of Developmental Studies 
programs will, ir part, be influenced by the 
interpretation of emerging data. 

Conclusions 

Assuming that the "Criteria" are met 
favorably, then Developmental Studies pro- 
grams will eventually change in purpose, con- 
tent, and scope. Dwindling numbers of 
students seeking provisional admission could 
mean that Developmental Studies would be 
gasping its last breath a few years from how. 
As an alternative to this situation, the larger 
institutions could raise their standards again, 
artificially creating a need for developmental 
programs. However^ the hew high school cur- 
riculum will hot guarantee that all of its 
graduates will qualify for admission to the 
university within the system, although many 
will qualify for the colleges. From this per- 
spective } it seems likely that the flagship in- 
stitution would continue to operate its 
provisional admissions program while the 
smaller schools within the system may elect 
to discontinue such services. Most imme- 
diately, in an effort to remain competitive, 
Developmental Studies programs might at- 
tempt to create more integrated, more inter- 
disciplinary approaches to instruction across 
its three basic skills areas. 
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In preparation for the task of relating 
SACS' hew criteria to instructional develop- 
ment I decided to use my dictionary and look 
up the correct definition of a couple terms. I 
was quite in trigued with Webster's definition 
of the twms "implicate" and u \mpV cation " 
One definition stated that this means *_ to fc^M 
or twist together. 99 Ah alternative ^ defihiiioh 
was "to bring ;Jnto intimate ^or ^Incriminating 
co J in ? ct ' ori;M another approach to defin- 
ing this concept was "a logical relationship 
between two propositions;" These definitions 
were interesting and several thoughts came to 
mind in thinking about how SACS' "Crite- 
ria" should relate to instructional develop- 
ment in higher education. I did not find a 
definition for "instructional development" 
and so, had to construct one myself. I define 
instructional development as the sum total of 
attitudes, resources; knowledge, and activities 
embedded in ah institution thai when mobili- 
zation can be used for the pre notion of ef- 
fective teaching. It is my hope, then,- that our 
discussion on this topic will allow us to take 
some of the SACS' "Criteria" and examine 
their relationship to instruction in higher 
education. 

There afvs numerous criteria in the hew 
SACS' docvsr.ni thai relate, directly or in- 
directly, to instruction. I have selected six 
areas thai f.tlnnh are pertinent for today's 
discussion. T«ie following paragraphs will con- 
tain some of jiiese connections. 

Institutional Purpose, Planning and EvaSaa- 

t ? ~r 

: have been impressed with Kenneth 
Ebel's account in "The Craft ci Teaching" of 
how a society can best promote valued activi- 
ties. He says that if he wanted to restore the 
Druid Priesthood the most effect?*- i means 
would be to build forests in which t-oey could 



live— rather than offer a "Drajd of the Year 
Award." Ebel's analogy h most appropriate 
wlthjn our^ontext The most effective way to 
promote good teaching is to have institutional 
goals and reward^ that reflect die value of this 
acti vl ty ; SACS^gujdel|nes u nder Secti Ion HI, 
Institutional Effectiveness, urge us to attend 
to procedures for planning and evaluation. 
This is where we should begin with respect to 
instructional development, { would say that 
now Is the time to examine the following: 

♦the changing nature of our society 

♦the changing nature of our students 

♦the changing nature of bur disciplines 
and professions 

♦the values held by our social institu- 
tions that relate to teaching and learn- 
ing 

♦the creation, and maintenance of^ an 
environment and ambience that pro- 
mote rigorous _ inquiry, high quality 
scholarship, and ah enthusiasm for 
learning together. 

Sh short, for there to be serious instruc- 
tional development within an organization 
there first must he serious planning L This 
planning should contrlbtiie both to the in- 
structlc^al goals of the institution as well as 
bt3ttfne_ strategics for evaluating instructional 
outcomes; 

Undergraduate Instruction 

As stated, undergraduate Instruction 
ihouU! Sow from the goals of the institution. 
SACS gMideiines make it clear that course 
goals and objectives should be formulated and 
maoe public to students at the outset of any 
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educational experience. Instructional methods 
should [ be comprised of those techniques 
ffiat are most effective in producing the de- 
sired learning outcomes. For example, instead 
of sdways^telHng students how scientists op- 
erate, perhaps it would be better to provide 
them with i opportunities to get involved in 
science-related behaviors* The evaluation pro- 
cedures used in a given course should measure 
*5_^ s j f ' e ^L^ u ^ c 9 m ? s of that course, and this 
information should be used to improve 
instruction; 

Graduate instruction 

The heart and core of good graduate in- 
struction is bringing together in a facilitating 
environment active, productive, and ^compe- 
tent scholars with enthusiastic^ motivated, 
and capable students. Graduate faculty mem- 
bers jhquld not only stimulate creative in- 
vestigation of ideas, but should be accessible 
to their students and serve as responsible role 
models in their disciplines or professions. 

Faculty Development 

Institutions must provide opportunities 
for continuing growth and renewal of their 
faculty. The movement of faculty from in- 
stitution to institution and the influx of hew 
faculty into academe that characterized the 
1960s and early 1970s ho longer serves as a 
natural source of renewal. At The University 
of Georgia, and probably this is_ not unlike 
other institutions, two-thirds of the faculty is 
over 40 and one-third is over 50. This "gray- 
ing" of jhe faculty combined with the fact 
that students are perhaps less prepared, yet 
more sophisticated in otherj^ys^frbm years 
P^?_P r 0 ^?? 65 . 311 . ^ er- wi den m g jpp between 
teacher and student This presents problems 

ports sound alarm that we may hot be able to 
attract and retain in the future the quality of 
professoriate that we have enjoyed in the 
past. 

Faculty development should be a con- 
tinual process with equal attention being paid 
to early career, mid career and late career 



faculty. Each of these groups has special 
heeds and colleges and universities benefit by 
having programs that address these needs. 

Instructional Resources and Services 

As the instructional process becomes 
more complex the more important it will be 
for institutions to provide faculty with ap- 
propriate resources amd services. The guide- 
lines in the hew ^A6S document specifically 
address many Important resources and in- 
directly suggest others^ Some of the resources 
and services that should be provided are: 

♦audiovisual and duplicating services 

♦instructional design assistance 

♦small grants 

♦assistance with external funding 

♦computer literacy workshops 

♦testing and evaluation services 

♦learning centers for students 

With the advent of computer assisted 
instruction has come the discovery that it 
takes between 50 and 200 hours of work to 
produce one hour of high quality instruction. 
If faculty do convert portions of their curri- 
culum Into software delivered systems, insti- 
tutions will heed to provide the kind of 
technical assistance necessary to make this 
practice feasible. 

Support Services for GTAs 

One of the most noticeable additions to 
the new SACS "Criteria" has been the 
attention paid to the qualifications of 

graduate teaching assistants. Graduate 

students who have primary responsibility for 
teaching a course are required to have 18 
semester hours of cbursewbrk in that area. 
With increased concern over^ the question j>f 
how much instruction should be delivered by 
graduate students will come an increased need 
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for institutions to provide training, 
supervisory, and evaluation programs for 
GTAs. This investment not only leads to 
immediate gains for the institution and its 
students but serves to help launch the teach- 
ing careers of faculty, hence serving as a 
major influence on the career-long instruc- 
tional skills of an individual. 

Summary 

Many recent reports have urged colleges 
and universities to pay closer attention to the 
undergraduate curriculum and to improve 
undergraduate instruction. Trailsigns point to 
the fact that instruction in colleges and 



universities will Receive increasingly more 
attention during the remainder of this century. 
Challenging times lie ahead and the hew 
SACS' "Criteria" suggest that more attention 
wjll be paid to faculty characteristics and in- 
stitutional support of instruction in future 
accreditation endeavors. This is an excellent 
time for institutions to evaluate where they 
are and where they want to go, and the SACS' 
hew "Criteria" can be a valid guide to use in 
this process. 
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IMPLICATION OF SACS' NEW CRITERIA 
ON COLLEGE ADMISSIONS 



John W. Albright 
Office of Admissions 
the University of Georgia 



Allow me to address you this morning, 
if you will, hot ±s a struggling student of 
American higher education. Instead, I would 
like to reflect on the Southern Association's 
hew mandate for planning and evaluation 
from my role as a practicing admissions pro- 
fessional—a footsoldier, as it were, involved in 
the daily struggle among the trenches of our 
college recruiting wars. The few observations 
I would like to share with you this morning 
were put into clear focus last week from a 
meeting I had been asked to attend. 

Last Tuesday I spent the evening with a 
small group of parents at a nearby public high 
school outside of Athens. They called them- 
selves the Academic Boosters Club. The 
school's principal had invited me to share 
with them any insights they could use for the 
promotion of academic excellence among the 
students of their school that I had picked up 
with my work promoting the University of 
Georgia's Honors Program and our freshman 
scholarships. 

One of the word pictures that I painted 
for them involved a sight that is now common 
across America in the fail: that is the event 
called the College Fair, where well-dressed 
admissions representatives from upwards of 
250 colleges stand behind, tables that are 
groaning under the weight of stacks of glossy 
brochures from their institutions while hun- 
dreds or even thousands of high school juniors 
and seniors and their parents shuffle by. We 
usually hold these fairs how in what has 
become the mecca of American teenage social 
life— shopping malls. The symbolism of these 
events being held in shopping malls, I told the 
Academic Boosters, is overwhelming: High 
school students today, in this era of the 
demographic slide, are Buyers in a Buyers' 
Market College-bound adolescents, espe- 
cially those blue chip scholars with 



exceptional academic promise similar to the 
sons and daughters of the Academic Boosters, 
are rare commodities that we in higher 
education covet Stand back, I said, and 
watch all the promotional literature being 
sent to your sons and daughters fill up your 
mail boxes. Take advantage of the pheno- 
menon Mleyoucan. 

This image seemed to please the Aca- 
demic Boosters Club, it launched us then 
into a discussion of many hew no-need 
scholarship programs that colleges are creating 
as part of their package of recruitment. The 
Boosters' eyes started to sparkle. We also 
talked at length about picking colleges that 
offer the right combination of opportunities 
for students, and _ how*, in my opinion, most 
students could do well at any number of 
different colleges. 

Later in oiir discussion 1 described the 
tougher high school curriculum guidelines 
that all colleges, even junior colleges, in the 
University System of Georgia will require of 
entering freshmen in 1988. This has become 
a hot topic. Admissions colleagues from 
several Southern states have telephoned me 
the past few months to find out how the 
University of Georgia's admissions procedures 
will change to carry out these new require- 
ments, because their states were enacting 
similar legislation. 



A gray-haired father interrupted me at 
this point "Why," he wanted to know, "if 
you're all having more and more trouble 
getting students each year, are you raising 
your admissions requirements? It seems to 
me that you'd want to lower them." 

This Academic Booster was obviously 
also a student of the American free enterprise 
system. My answer to him was this: The 
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effect of these legislated admission standards 
may well decrease bur number of freshmen in 
the public colleges and universities of our 
states. I was riot sure, even though my office 
had been studying the matter, and I do not 
think anyone in our state is right now. The 
intention of these standards, though, was that 
these freshmen would be better prepared to 
Handle bur academic college curriculum from 
having been forced to take traditional college 
preparatory courses in high school. We might 
have fewer freshmen, but more of them 
wou!c._make it all the way to graduation arid, 
1 cop.cIlv ti, we hoped ib have the same total 
erirc-imehti I finished with a note of opti- 
mise u/ suggesting that we might eventually 
have even more students, overall, as a result of 
these new standards. 

I repeat this story ior you this morning 
riot as a way of dis;^ r mg "The Nation at 
Risk," or new admissions retirements, o r the 
content of the g?nerat education curriculum. 
Rather, my evening with Academic 
Boosters illustrated three things about the 
actual practice of admissions officers at this 
point in our history, arid how these three 
activities may change as our colleges fulfill the 
spirit bv planning atid evaluation that Section 
III of the Southern Association's new 
"Criteria for Accreditation" calls for. 

The first point is that we admissions 
officers are essentially communicators. That 
was my role at the meeting of the Academic 
Boosters. Sometimes bur communication 
involves the mundane. The telephone ririgs: 
Have we received Sally's hi^i school, tran- 
script yet? When does fall quarter start? Can 
freshmen have cars? How much is tuition? 
Do you have a double major in music arid 
marketing? We do what bur catalogs or the 
Peterson's Guide books do for the college 
selection process, but we are able to do it in a 
more persona] way. 

Often bur communication is more 
complex. We explain policies— like the new 
1988 guidelines-: and we discuss trends— like 
scholarship recruiting. Here are two typical 
scenarios that I find are typical in the college 
admissions business: We meet with twenty 



seniors in the high school library, or we sit 
down in bur office with mom and dad arid 
their daughter, all dressed in their Sunday 
finest. In each scenario we begin by dis- 
cussing the mundane details of admission arid 
enrollment at bur institutions, but eventually 
the conversation deals with more basic issues. 
The underlying question, which hone of us 
ever asks out loud, becomes, "What will I do 
when I grow up? Tell me how your college 
can help me do that. 11 

Our second role, then, is to be college 
counselors. Our jobs put lis in counseling 
situations on a daily basis, arid both our in- 
stitutions arid pur shared humanity demand 
that we help all that we can. This is a tradi- 
tional view of the function of admissions 
officers. In fact, we are usually called ad- 
missions counselors. But our roles as coun- 
selors are sorely tested in the era of the 
Buyer's Market. Imagine this statement: "I 
know we have a hundred fewer fireshnieri this 
year," we might try to tell bur college presi- 
dents, "but we in the Admissions Office 
helped a hundred more students find just the 
right college for their future development" 
JVe all know that argument is doomed. 

The filial role of admissions officers is a 
newly developing brie. It involves recruiting 
students, which of course will be our major 
job on into the 1990s while we wait for the 
next baby boom to mature to adulthood. 
This emerging role involves communication 
arid counseling, too. The common term that 
people have been using to describe this role 
has been "marketing." Marketing is too 
narrow r concept, though. It has too crass a 
business connotation for many people in 
ivory towers, so new terms are coming into 
use. One new buzzword that describes this 
concept is "enrollment management" This 
concept has major implications, I think, for 
the function of admissions officers under the 
requirements for "outcomes assessment," or 
whatever the Southern Association eventually 
would like to call the planning for and eva- 
luation of institutional effectiveness. 

Under the concept of enrollment 
management, admissions officers and those 



73 



Albright 



73 



who set admissions policy become part of a 
large management team made up of faculty 
and administrators from all areas of an Insti- 
tution. These team members come to manage 
enrollment by recruiting the most appropriate 
students for their institution, given its re- 
sources and its educational objectives. They 
give the recruited students appropriate aca- 
demic, career and personal advising and 
counseling; and they provide these carefully 
selected and groomed students wi^i appro- 
priate academic course work and extracur- 
ricular services and stimulation. Enrollment 
management, carried to its logical conclusion, 
includes career planning and placement and, 
eventually, alumni development and fund 
raising. The admissions officer recruits stu- 
dents with an eye toward their becoming 
active alumni some day. The alumni officer 
develops alumni clubs with ah eye toward 
their helping to recruit students and to sup- 
port the educational growth of those students 
once they are enrolled. And everyone, from 
groundskeeper to president, evaluates their 
effectiveness in relation to the overall policies 
o f the institution's enrollment management 
goate. 

This concept of enrollment manage- 
ment focuses en the students. Unlike the 
marketing concept that we might borrow 
from the business world, though, enrollment 
management does not change the product to 
fit the real or imagined needs of the market. 
Enrollment management instead forces ail 
members of a college community to form 
consensus about what direction their joint 
educational venture should take, and then 
asks the admissions officers to recruit the 
students most appropriate for that venture, 
lie dynamics of the situation keep the ad- 
missions officers from simply becoming sales 
representatives. They help define their 
institution. They help filJ it with students. 
And, by doing their jobs honestly and well, 



they help assure that the students they helped 
recruit actually graduate. 

Regardless of enrollment management 

or any other such trendy concepts that come 
along though, colleges who commit them- 
selves to an asse -ment of the outcomes 
of their labors then logically must first assess 
their input Admissions officers stand at the 
entrance of their institutions, sometime quite 
literally. It logically follows that we are in 
the best position to assess, or to help with the 
assessment at least, of the raw products of our 
industry. We talk to freshmen first We 
encode the data from their admissions appli- 
cations. We are the first to connect names 
and faces. Admissions officers today are 
forming closer working relationships with 
student data base managers and institutional 
researchers because of their primary prox- 
imity and first name familiarity with this raw 
data. 

At the same time as we offer assistance 

to this growing need for the gathering of basic 
data, we are gathering our own kinds of data 
in our roles as communicators and counselors 
and recruiters. More than most public 
relations officers, and almost as much as most 
presidents, people in admissions offices are 
daily called on to explain facets of their 
institutions. We must describe statistics 
in human terms that teenagers and taxpayers 
can understand. We Must turn policy and 
data into brochures and videotapes and 
campus tours. 

That is what we dp in the admissions 

trenches. That is the challenge that f and my 
colleagues face. That is why I read Section III 
of the SACS "Criteria" and get excited. I see 
another instance of how admissions will fin- 
ally break out of its salesmanship stereotype 
and legitimately become part of the larger 
scheme of management in higher erf cation. 



74 



75 



THE IMPLICATIONS OF THE NEW CRITERIA FOR ADULT LEARNERS 

Margaret t: Holt 
Adult Education Department 
The University of Georgia 



On October 16, 1985, Dr. James 
Rogers, Executive Director, Comm ission on 
Colleges, Southern Association of ^Colleges 
and Schools, presented the keynote address at 
the National University Continuing Education 
Association's Region j IS meeting in Knoxville, 
Tennessw. He titled his presentation "Criteria 

for Accreditation: A Tool for a Future of 

Excellence^ for Continuing Education L ^o* 
grams in the Sooth;" In December I phoned 
jim to ask if I mightuse his remarks L ffom thjs 
meeting as a L*prinj^oar<^ f or my comments 
today. He most kindly aj^eed.JJverall, jim's 
comments reflect a belief that the new 
"Criteria" will improve the quality of con- 
tinuing education. I would suggest that thisjs 
possible Sf rte new "Criteria" function as 
finely gauged precision tools unlike the more 
3tendard tools fonrieriy empjoyed to assess 
continuing education. (Perhaps thisjs why the 
accreditation gu ide is n ow_ termed "criteria 
for accreditation" rather then "standards"^) 
New^ tools for measuring all educational 
programs must be more sensitive anc precise 
b_ ec ?ose JO? Jieterogeneiiy and onique 
features of ihe many new audiences served by 
higher education. 

Nearly a year ago. Dean A[ Buccino, 
^P 17 ^9i'5I e _of_ _Edncattion at the 

yn!X?f?l?y of Georgia, commented to the De- 
partment Chairs of his College, "We are wit- 
nessing a shift in society about education 
f r _9m a _ preoccupation with inputs to a 
preoccupation with outpjjts." j agree with 
this remark and would add that the new older 
students on our campuses and the economic 
climate of our country are two major factors 
contributing to the shift 

Related to these factors of emphasis on 
outputs and economic pressures, listen to 
some of jim Rogers' comments in Knoxville 
about ti?e recent revision of standards: 



"The ^comm|ssibn recently completed a 
revision of the standards and developed 
a new emphasis which has as one of 
ite major components, tiie measure- 
ment of tfie^ffectlveness of an Institu- 
tion^ proems. The_ ^Criteria for 
Accreditation^ a document which has 
restructured the standards to include a 
major new emphasis on planning and 
evaluation, will move our efforts to a 
new level of effectiveness, especially as 
they relate to your area. The 'Criteria' 
will serve as a tool for achieving this 
goal by improving the quality of your 
programs and ascribing to these pro- 
grams a legitimacy equal to any In the 
academic arena. But you may ask— why 
the shift? Weren't the old standards 
acceptable measurements for identify- 
ing quality education programs? The 
answer is: PARTIALLY. The old 
standards focused exclusively on in- 
stitutional resources and^rocesses and 
operated under the gumption that if 
an institution had certain resources and 
used certain processes, effective educa- 
tion would occur. However, L new non- 
trad iti onaj jn sti tu t ions, acce p ted i n t o 
tiie^ membership, spurred new ^ thinking 

^cS^diDS _ n PJ 1 ^^'^ na j_P r ^8 rat r i i an< ! 
nontraditicnal means of learning We 

could no longer be comfortable in our 
outdated images of higher education. 
We could no longer evaluate orograms 
along traditional lines because we could 
hot always evaluate traditional re- 
sources and processes. _At__th_at point, 
the College Commission began to ad- 
dress the quality issue. Question.* we 
were asking of nontraditional programs 
seemed equally appropriate for tradi- 
tional programs. The approach finally 
adopted wa; to add to the previous 
reliance on consideration of inputs, a 



75 



76 



Holt 



significantly greater emphasis bri the 
evaluation of institutional effectiveness 
on the results of the educational pro- 
cess. As we developed the idea, com- 
mittee members suggested that both 
measures be used concurrently— that we 
evaluate inputs and processes as well as 
the results of these processes." 

Allow me to carry on with Jim's com- 
ments a bit further. 

"In conjunction with this hew direction 
of looking at the effectiveness of an 
institution and, in ftim* redefining qua- 
lity, stiidt':n& also redefined their con- 
cepts arid attitudes about academic 
quality. In 1977, students defined aca- 
demic quality in terms of institutional 
characteristics:, quality of the faculty 
and quality of the student body as 
determined by SAT scores (inputs). In 
1985^ students. defined academic qua- 
lity in terms of 'after college' criteria: 
i.e., the number of graduates gaming 
acceptance to professional schools and 
the number securing desirable jobs in 
business and industry. Not unlike the 
descriptions of quality outlined in the 
'Criteria/ students' concepts of aca- 
demic quality shifted from descriptors 
of the college to fie outcomes of an 
educational process, Consequently, it 
was no surprise to find that t?ie Carne- 
gie Foundation for the advancement of 
teaching conducted a study tracking 
th<? undergraduate major. Among other 
things, it found that 'American under- 
graduates in their choice of majors are 
increasingly preoccupied with prepara- 
tion for jobs.' Higher education is 
'labor intensive',— students look more 
to the practical, material benefits from 
college— the outcomes of the educa- 
tional process. Consequently, a part of 
the new emphasis of the 'Criteria' re- 
flects this e ideational priority of 
society. Ail of higher education will be 
better served if these risw 'Criteria' ad- 
vance a more intense look at ouputs, 
and continuing education is no excep- 
tion." 



Iii summarizing these initial remarks 
from Jim Rogers' address to this region's 
members of the National University Continu- 
ing Education Association, I wish to under- 
score three issues: 

1) Economic and demographic shifts will 
drive higher education's decision mak- 
ing about what types of services will be 
offered by^ the higher education com- 
munity. The implications for adult 
students are even more preoccupied 
with jobs and job shifts than their 
younger counterparts on-and-off-cam- 
pus. The almost overnight creation of 
eighteen corporate colleges reflects the 
societal recognition that lifelong learn- 
ing for survival in a technological 
world is imperative. 

2) The emphasis on outputs from both 
traditional programs will heighten again 
due to economic concerns and priori- 
ties of students in the 1980s and 1990s. 

3) The tools of measure outputs, thai is 
results of training, learning transfer, or 
behavioral changes related to education 
are generally crude. Accrediting bodies 
are going to be challenged to find in- 
dividuals qualified to define what out- 
puts "should be" expected in both 
traditional and nontraditional programs 
and definitely exasperated in locating 
individuals capable of conducting out- 
put studies and assessments. To be as 
precise as passible, the new evaluative 
tools will need to incorporate both 
quantitative and qualitative method- 
ologies.The state of the art in qualita- 
tive evaluation is primitive and the pool 
of individuals skilled in preparing, 
managing, and interpreting qualitative 
evaluations is miniscu le. 

Dr. Rogers' told us in Knoxville: 

"In the new 'Criteria,' as you no doubt 
know, continuing education Is incor- 
porated into the major section on 
educational programs. It is no longer an 
entity unto itself. The 'Criteria' main- 
streams the special programs and 



76 



Ho't 

elevates them to an equal level along 
with undergraduates and graduate pro- 
grams. It s very placement in the 
document— preceding faculty require- 
ments and following graduate pro- 
grams—indicates that is viewed as a 
stantial program area in higher educa- 
tion and is deserving of the same level 
of attention a c all other academic acti - 
vities of ah institution." 

I personally have mixed emotions 
about the implications of this. placement of 
continuing education. First, if the separate 
placement for continuing education in the old 
standards meant that it was percer*d as an 
"add-on" in higher education (Jim's words), 
then certainly incorporating it into the main 
body of the "Criteria" is improving the 
perception of its worth in the higher educa- 
tion community. However, if the new incor- 
porated placement will mean that continuing 
education will be forced to adopt traditional 
practices that have proven insensitive to some 
of the unique heeds of nontraditional adult 
audiences, then perhaps a "separate-but- 
equal" standard criteria would be more 
desirable. However, Jim's comments are re- 
assuring that the hew '/Criteria" will embrace 
creative approaches to instruction: 

"The •Criteria' places equal erf ohasis 
on process and results, freeing continu- 
ing education programs to address 
concepts of educational delivery and 
hew uses of technology in teaching. For 
example, it allows for new concepts 
such as telecourses— courses which 
embrace large numbers of new and 
different clientele. Such innovative 
instructional methods were not clearly 
provided for in the old standards. The 
'Criteria' encourages institutions, and 
continuing education progams in par- 
ticular, to be even more creative in 
developing hew ways of reaching po- 
tential learners. . ." 

Although, I have suggested there is a 
tremendous challenge, facing accrediting 
bodies on the matter of evaluating outputs of 
educational programs, I applaud the commit- 
ment to do so. My own research related to 
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outputs of continuing education programs 
indicates that we must intensify our efforts to 
assess transfer of learning from educational 
programs to work and other setting*. Al- 
though the general growth of continuing 
education programs for professionals indicates 
individual and organizational satisfaction with 
these educational experiences, there is 
mounting evidence that reports of positive 
impact of such programs on job oe^ior de- 
serve challenge. As professional organizations 
are more rigidly asked to establish the finan- 
cial creditability of such programs, partici- 
pants' attendance and self-reports of 
satisfaction ar~ counting less as measures of 
actual program value. Questions about on-the- 
job performance change following programs 
or learning transfer from courses to tasks are 
increasfngly raised, and in a few situations the 
continuation of programs threatened when 
sponsors contend they cannot justify costs. 

If the new _ "Criteria" influence the 
people, in continuing education to conduct 
more follow-up or impact studies of their 
programs, then* in my opinion, the implica- 
tions for adult learners will be extremely 
positive. Unfortunately, there is considerable 
evidence in recent literature that learning 
transfer from program to post-program 
settings often fail. Although learning transfer 
from adult education programs to changes 'n 
performance and practice following programs 
may be a central concern of many program 
planners, impact studies have not been widely 
implemented. That is, the commitment and 
energies of those who develop programs has 
been most often directed toward what 
happens at a program, not after. The timing 
and placement of most program evaluations 
illustrate this emphasis on the program itself 
rather than knowledge utilization following 
instruction and training. Moss, Blyt!ie, and 
Barfield (1978) contend such post measure- 
ment yield only a "happiness quotient." 
Continuing education professionals know it is 
an all-too-common practice to enclose an 
evaluation form in the packets of program 
participants and ask that such be completed 
before individuals leave the program site. It is 
reasonable to argue that this is rot an 
effective evaluation practice as these f antici- 
pants will be influenced in their reporting by 
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such factors as fatigue, and anxiousness to go 
home, and/or a post-program eapfioria from 
meeting new people, old friends, being 
stimulated by new ideas, or a jaded disregard 
for the worth of completing "one more" 
post-program evaluation form. The informa- 
tion yielded on such forms frequently denotes 
degrees of satisfaction related to content 
design, presenters, and other program factors. 

The literature is fujj of studies Sat 
report people's levels of satisfection or 
self-perceptions of continuing education pro- 
grams. Most studies report_positive satisfac- 
tion, but studies that go^ the next step and 
attempt to correlate reported satisfaction and 
self-perceptions tend to show little correlation 
with actual knowledge utilization^ following 
the programs. For certain programs, pre-and 
post-tests are conducted whiclr additionally 
provide programmers wjth^ information about 
cognitive gains resulting from program parti- 
cipation. Clark (1981) for example, states 
that cognitive ^testing is the primary means for 
demonstrating accountability to the public re- 
sulting from nursing education, ©nee this 
information is tallied, those accountable for 
program justification often make quantum 
leaps of faith by suggesting their programs 
had results, outcomes, impacts, or effects on 
[qb performance or behavior of participants 
following programs It would be advisable for 
continuing educators to refrain from such 
lofty leaps and review the studies that 
demonstrate a lack of evidence to support 
correlations between program satirfaction 
and/or cognitive gain, and direct impact on 
performance or behavior change following 
programs, in fact, several studies show no 



correlation between satisfaction and/or cogni- 
tive gain and performance. Undoubtedly, 
affective and cognitive gains will continue to 
be measured and reported by program eyal- 
uators, but as accountability and cost-effec- 
tiveness attain greater importance to profes- 
sional organizations, continuing educators will 
be expected to design programs with results 
that are transferred and lasting. The hew 
"Criteria" should help facilitate actions to 
evaluate learning transfer and thereby 
improve the quality of continuing education. 
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At the offset j must express publicly my 
appreciation for the skill employed by Jlr; 
Rncher and his associates Jn designing tnis 
conference. His success is u faceted by the 
composition and complexion of the various 
segments of the program. Thus, I [am exposed 
again, ; as I have been jf or the past 1 \9_ years of 
association with him on this campus, to 
Cameron's peerlessness ]n^ working effectively 
with people and in his attending to all details 
in his planning and execution, , Some of us 
have said many times during the years that 
Cameron always covers all bases. He has done 
it Jn LP' an njnS this cqnference-e thnic consider- 
ations, gender considerations, senior pro- 
fessionals, younger professionals, varied 
interests incl uding vocational considerations- 
and this is typical of Cameron. 

As j look at this audience, I see much 
more heterogeneity than I have seen typically 
in other conferences in this room. That is just 
the way Cameron ^incher works. Cameron, I 
join others in thanking you again this 
morning. 

With great interest and Inevitable 
recollections, j have listened to the pro- 
ceeding speakers during Jhis valuable con- 
ference. Now Jt is my tarn to offer some 
comments on the assigned topic: "impfij 

cations of J^ew Criteria for Higher 

Education's Changing Clientele". Having 

announced the topic I [shall proceed as the 
proverbial minister who intones solemnly his 
topic, presents his scriptural basis, and then 
departs from his topic not ta return until it is 
time to "open the doors of the church" to 
receive new members. 

I Hstened with Interest to Ron Simpson's 
presentation of definitions of "implications". 
Cameron said that my responsibility, also, was 
to discuss implications. Ron, after checking 



his dictionary^ defined implications as 
"twisting or bringing together or working in 
such a manner to tie some elements together 
as you see them". I now shall proceed to 
"twist". 

There is an advantage to being one of the 
jast speakers because by this time everything 
has been covered. Margaret Holt just spoke 
about "implications of the Criteria for 
Learners." Obviously ' too am speaking 
about learners. j am concerned about 
learners. Margaret has covered all those fine 
academically defensible points regarding 
learners. So I can just be Herman Smith as 
Cameron knew I would be and speak to you 
in terms of some concerns that j see and feel. 



"Higher education's changing clientele"™! 
did not have the privilege of discussing this 
topic with Cameron, so I cannot be positive 
about what he had in mind when he devised 
it. However, I have pondered these words 
which are, in my judgment, a type of 
euphemism. "Higher education's changing 
clientele", I conclude, focuses upon those 
segments of our society which have not been 
served very well in the p?st: ethnic 
minorities, older citizens, citizens with various 
physical handicaps, and, of course, people 
who suffer serious economic disadvantage. 
But changing clientele bring to higher 
education new or different perceptions about': 
many specifics. They bring often diffes^nt 
backgrounds, life experiences, and styjes; 
they bring additional skills and potential as 
well as needs and major interests sometimes. 
There is a difference; there must be when the 
new clientele arrives. All of these elements 
are legitimate and worthy of consideration in 
the higher education experience. But at the 
same time, the changing clientele bring much 
that other traditional arrivals bring to the 
higher education scene. 
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So there is a difference and at the 
time ^ere is not a difference, and it seems to 
me that those of us who are responsible for 
planning and delivering academic service 
should understand that and govern ourselves 
accordingly. 

I have pondered the various presentations 
made jiuring this conference in the light of 
demographic and other information to which 
I have been exposed during the past few 
weeks in the course of my professional 
responsibilities, j have reflected further upon 
two statements made earlier by two previous 
speakers. Dr. Rogers of the Southern 
Association of Colleges and Schools said at 
dinner last evening, "the winds of change are 
blowing in higher education with brutal 
force." What a striking statement! His pro- 
nouncement suggests serious implications. As 
one dedicated to being a catalyst, j personally 
applaud those winds. He implied that the 
winds are surging and raging, not blowing 
gently. 

Second, I continue to reflect upon the 
enumeration by Tom Redmon of top positive 
factors that help a learner to achieve the out- 
comes stated. J"om listed caring attitude of 
facu Ity and staff, high quality teach ing, and 
then he also talked about involvement in 
various campus activities, j thought about 
myself in terms of who i am, who j have 
been, where j have been, where j am this 
morning and where j might be. If all of you 
are looking up here, you can see me, you are 
able to interpret the statement. j have 
responded to all of this as I think about how I 
happened to enter college as the first one in 
my family to enroll in college and graduate. I 
and others similarly situated presented our- 
selves to the college officials not knowing 
very much about all the theory of higher 
education and the criteria, standards, etcetera, 
but the teachers let us know that the assigned 
educational tasks , must be accomplished. 

We were very much impressed with the 

importance of being in college. Then the 
teachers proceeded I to let us know that we 
would achieve those objectives, and that we 
could achieve those objectives. Then they 
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said, "I can help you" and "I will help y&.? % 
Early jn our college career, then, a favon Ole, 
supportive climate for learning and achieving 
was established, [ knew that { would be there 
for die ride and that I would make it to the 
finish line, j think this is what Tom Redmon 
was speaking about. 

I would state that one specific on his list 
was not present in my situation. He said 
"adequate financial aid." Tom, \ did not have 
any financial aid, I did not have any money, 
but j did the best I could, j am able to testify 
publicly this morning that I was so busy 
trying to earn the tuition and be able to pay 
the costs the coming Fall that I did not have 
time to be poor. I did not have time to be 
disadvantaged; and I did not even have time 
to be different, j was just a college student 
with no money who had been inspired by his 
high school teachers to enroll and accept the 
tiny scholarship offered, no financial aid. I 
concluded, once enrolled, that j must get 
busy and I had better get busy, and so j did. 

I am very conscious of the fact that this 
conference by design is focused, or is 
supposed to be focused, upon higher 
education in Georgia, j assume that if j were 
in a conference that was focusing upon higher 
education in Wisconsin or Oregon or Iowa or 
Idaho, there might be some different con- 
siderations before us. 

j am informed that in one sense there are 
really two Georgias: urban Georgia and rural 
Georgia. One Georgia is basically or compar- 
atively rich; the other Georgia is poor. You 
have read as I that the state's urban counties 
contain only 5.3 to 15.2% of the population 
below the poverty level, while in rural Georgia 
between 19.5 and 40.796 of the people live in 
something called poverty. Yet all Georgians 
are individuals of potential and ability and all 
Georgians deserve the chance for a higher 
education if they desire or see the light How 
will the opportunity and the need-and we do 
have the need-for all Georgians to have their 
potential developed be addressed? How Will 
that need be achieved? When the factor of 
race is added to the above, a challenge is 
compounded markedly. 
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Georgia's more than one million public 
school students, as I read, are the tenth largest 
number or assembly of JtUderits in the 
country, but as of 1980 only 5696 of 
Georgia's residents were high school grad- 
uates. _ Oh this dimension bur state is 43rd 
from the top of this ranking nationwide. So 
there is much that must be done here to serve 
bur people. We will have people who heed to 
be educated for years to come if we can 
decide to respond to their needs and their 
potential. There will not be a shortage of 
people. 

Many details which surfaced during my 
recent trips to Washington kept swirling 
through my consciousness as the proceeding 
speakers have continued. _ For example, in 
discussions in one office of demographic data 
;t was pointed but that 29% more black 
students graduated from high school in 1982 
than in 1975, but black enrollment in college 
dropped 11% during the period. Why? What 
is today's trend and is that trend still 
observable? High school graduation rate for 
Hispanics increased 38% during the 1975 and 
19*3 period while Hispanic college enroll- 
> y Vcl : ned 16%. Are we doing better here 



We are talking about Americans and 
Georgians and citizens who heed the benefit 
of higher education and who have potential 
for succeeding. I believe, whetfiet or not it 
has been expressed very well, deep down all 
these individuals do have an aspiration to be 
successful. 

Nationally I read that one half of all 
college students enrolled this academic year 
are holding jobs, a proportion that has risen 
steadily over the last 25 years. Many factors 
help to account for this phenomenon. One 
factor has to be the increasing diversity of 
clientele pursuing higher education and that 
report has many substantial implications then 
for the higher education experience. 

Finally* j have reflected upon many 
individuals whom I have known who despite 
all the odds have made their way to receptive, 



apparently capable, institutions of higher 
education and who have succeeded in 
accomplishing the set task. I have mused 
r ' ing these hours as to how this was done 
and the benefits to be derived if the explan- 
ation for these success stories could be 
distilled, collected, organized, understood, 
interpreted and then internalized. We might 
move ahead and provide more adequately for 
the climate and services heeded in Georgia to 
make our educational efforts more effective. 

I reflect inevitably upon the slogan that 
has been used at Atlanta University through 
the years: "I will find a way or make one." 
The expression reflects a determination to 
succeed in providing appropriate educational 
service for people despite all handicaps or 
impediments. 

The foregoing are among those thoughts 
and ruminations which have flooded my con- 
sciousness as I have observed the various 
presenters and followed their respective state- 
ments. The recital of my reactions and 
reflections provide a background for the five 
specific implications which I have distilled 
from this experience. I will merely recite the 
implications with little commentary altfco^iti 
each one provokes extensive discussion. 

1. The existing use and growing prominence 
of criteria is an established fact of life in 
hi^ier education. This is hot a lamen- 
tation by any means, but a statement of 
hope and optimism. I applaud the estab- 
lished fact I participate in many groups 
from time to time in which is expressed 
the views that "the cards are stacked 
against us", "they are no good", "the 
standards are not for us", "we can't 
do it", "lets attack them", etcetera, j 
don't join that crowd. The use and 
growing prominence of criteria suggests to 
me the need for a skillful, repetitive, wide- 
spread interpretation of tfie "Criteria," 
their purposes, their development and of 
the related implications. Thai, of course, 
is what Dr. Rogers was doing last evening. 
The process must be repeated again and 
again in all kinds of circles it seems to me. 
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The interpretations should seek, unfail- 
ingly, to communicate to all that a central 
concern of the effort is to strengthen in- 
stitutions and to strengthen their respec- 
tive positions and potential to provide 
needed service in their areas of operation. 
The heightening attention to the "Crite- 
ria" and standards suggests to me that 
institutions are enhancing their quality as 
they continue to pursue their respective 
objectives and institutional commitments. 

2. At the same time that justifible attention 
is given to administering the "Criteria/' it 
is imperative that adequate sensitive re- 
cognition be given to the human consider- 
ations or the subjects Who will receive the 
impact of the "Criteria." I am happy to 
say that I have arrived at a point where I 
never become so inebriated about the 
paper and the statements contained 
thereon that I really do not focus enough 
attention upon the reason for the rheto- 
ric-the people. There is a person in our 
midst who raised the question yesterday , 
"what about that human element, would 
you expand upon that?" We must always, 

it seems to me give, adequate attention, _ 
sensitive attention, to the human con- 
siderations. 

3. it would appear that any institution 
which commits itself to the attraction, 
preparation, and graduation of changing 
or new clientele would at time of making 
such a service commitment proceed to 
effect appropriate changes from its tra- 
ditional and historic patterns, designs and 
approaches to the task. The commitment 
to serving new clientele has far reaching 
implications for the institution. The 
institution then must change in every 
respect because the new clientele bring 
with them differences as I sought to 
enumerate earlier. 

4. The challenge of identifying, soliciting, 
admitting, preparing or serving, graduating 
and helping to induct the new clientele is 
so great and complex that a durable, 
dynamic, and creative <- '^boration of 
people and organ izatl ssary to 
enhance chances for e ems to 



me that the educational institution alone 
is hardly adequate for the task. At the 
Kettering Foundation we believe in tiiis 
collaboration-* broad-based cooperation 
that woiild include many people and 
many kinds of organizations. _ Only by 
such procedure can we realize the greatest 
benefit from or impact of the "Criteria." 
I perceive that Jim Rogers and his asso- 
ciates understand that and that is why 
they are interpreting and discussing the 
"Criteria", so widely in our state and 
region. This "reaching out" activity is 
critical. The process should include all 
the people back home, the ordinary 
taxpayers, the parent, etcetera. 

5. Finally, and I think this is a very 
important consideration, the hew clientele 
Should not be expected or required to 
assume total responsibility for extricating 
itself from the condition oi being hew 
clientele. All of us must help. We have an 
inescapable responsibility for so serving. 

Mine is art idealistic perception perhaps, 
but nothing less will serve me. Had this hot 
been so, j surely would not be here in this 
place today on this podium. An earlier 
memlk '' the changing clientele, a man who 
origins* j in Epworth, South Carolina, made 
a statement some years ago which is repeated 
again and again. I think the quotation is 
germane to my response to what I have heard 
and perceived at this meeting. The individual 
of whom I speak is the late Dr. Benjamin E. 
Mays. These are his words: "It must be 
borne in mind that the tragedy in life doesn't 
lie in not reaching the goal; the tragedy lies in 
having rib goal to reach. It is not a calamity 
to die with dreams unfulfilled, but it is a 
calamity riot to dream. It is hot a disaster to 
be unable to capture your ideal "-and we 
would strongly state that we must have them- 
"but it is a disaster to have no ideal to 
capture. It is not a disgrace riot to reach the 
stars; but it is a disgrace to have no stars to 
reach for. Not failure," said that departed 
South Carolinian arid member of the new 
clientele at the time, "But low aim Is sin." 
May we-each of us here assembled-in our 
future work as educators in Georgia be free of 
the sin of low aim. 
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The purpose of my remarks will be to 
suggest that it is inevitable that the Southern 
Association of Colleges and Schools 9 new out- 
crmes-focused "Criteria" for institutional 
accreditation will induce educational innova- 
tion. One might anticipate, on the basis of 
reviewing our experience with FlPSE projects* 
that the innovations will be many and far- 
reaching. That could be a very interesting* 
lively and constructive prospect 

Accreditation processes have been a 
force for the improvement of quality across 
the years* helpful to institutions and to con- 
sumers and promotive of public credibility 
and trust. The notion that the legitimate mis- 
sions of institutions can be diverse and many 
and that standards should be related to 
purposes* and effectiveness to objectives* has 
been a boon to innovation in this country. 

An exception is associated with be- 
yond-the-campus phenomena which are re- 
cent. Input-oriented criteria* among them 
library resources* have been barriers to non- 
traditional education and perhaps especially 
to the distance learning made possible by 
modern technologies. Innovators viewed ac- 
creditation as a barrier to certain kinds of 
access at a time when the great problem in 
higher education was access. Now that the 
great concern is shifting from access to the 
quality of learning and includes a focus on in- 
stitutional effectiveness* on outcomes which 
might be achieved in many ways* and not 
only on traditional inputs* accrediting agen- 
cies have special opportunities for leadership. 

The shift to an outcomes focus— a focus 
on the learner— makes it possible to encourage 
a wider range of modes of learning and thus 
for accreditation processes to encourage both 



quai.xy and innovation more powerfully than 
ever before. 

Moreover and predictably* the new 
"Criteria 11 will be catalytic to innovation for 
at least the following seven reasons: 

1. The Southern Association's outcomes 
focus is part of an evolving movement 
which will reinforce the thrust of the 
Southern Association and provide a 
wider set of stimuli to innovation. For 
example, the NIE panel which pro- 
duced "Involvement in Learning" re- 
commended that measures of student 
growth and development be used as 
indicators of institutional and pro- 
grammatic excellence; 

2. The outcomes focus is an appropriate 
answer and arguably one of the best 
answers to the demand for improve- 
ment in the quality of education* and 
may be politically important in secur- 
ing support for higher education and 
avoiding undesirable intrusions by state 
governments; 

3. The outcomes "Criteria" are funda- 
mentally different from the old criteria. 
They change the assumptions about 
what is important* in effect putting n 
place new building blocks for wide- 
ranging planning and operations* 

4. AH institutions will have to adapt* each 
meeting the new expectation in its own 
way; 

5. The possible ways to meet the 
"Criteria" are many; indeed the South- 
ern Association's "Criteria" explicitly 
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indicate the the paths are many, and we 
may furthermore anticipate that the 
effects will extend, in a variety of ways, 
to faculties, departments, whole insti- 
tutions and even systems; 

6. This change should set in motion a 
continuing process with feedback, so 
that improvement and innovation will 
proceed in either stages or as continu- 
ing change. It is explicit in the South- 
ern Association's "Criteria" that there 
shall be continuing programs of institu- 
tional research, which can themselves 
be innovators. It is implicit in this 
change of focus that accreditors lock at 
process, not brie-time outcomes or 
snapshots at long intervals, but at the 
Institutionalization of process and 
feedback loops. 

7. Because implementation will be diffi- 
cult in terms of methodology, faculty 
arid counseling deyeldprrierit arid ex- 
pense, arid because implementation will 
be a mariy-splendbrev* phenomenon arid 
people will be looking for models arid 
hoping for helpful dissemination, there 
will probably be a need to create some 
kinds of incentive funders, whose acti- 
vity in itself will be catalytic for inno- 
vation. 

It seems to me that we have before us 
in these new "Criteria/ 1 which might seem 
innocuous or routinely common-sensical to 
some of its readers, some potentially powerful 
stimuli to change which portend national 
leadership for this region. While there is some 
risk of stumbling, there is also an opportunity 
to be the leader. 

One hesitates to over-iriterpi;et anything 
riot yet implemented, but as a political scien- 
tist, whose own field I? neavily concerned 
with constitutional engis - ring, i.e. the effect 
of changes in the rules upon behavior, I have a 
bias toward interpreting this kind of change as 
likely to be broad! / consequential arid even to 
have serendipitous effects. For example, when 
European political systems decided that the 
results of elections should be reflected by 
proportional representation rather than by 



declaring a single winner, there followed con- 
troversies about the mathematical formulas 
for distributing the seats and about the thres- 
holds for winning any, and there followed 
changes in the ballot, the ejection districts, 
the nominating process, the number of 
parties, arid legislative behavior, including the 
pattern of coalition governments. The whole 
system performed differently following this 
fundamental change in how to assign value to 
a process— in this case election rather than 
education. In parallel, anyone who constructs 
education scenarios analytically may work 
with one new independent variable but a 
whole flock of dependent variables. 

Thus I want to suggest the prospect 
that the systematic implications of the out- 
comes focus may be every bit as important a 
consideration as the methodological problems 
of measuring the outcomes. Indeed, I will be 
efaboratirig this point 

Scenario construction is not my func- 
tion today. I am part of this panel primarily 
tecause of my experience as Director of 
FIPSE, whose portfolio of projects, which 
range across all fields, have included a few 
real-world experiences pertinent to the impli- 
cations of meeting the Southern Association's 
criteria. The cases will probably be of interest 
to potential adapters and innovators, and will 
lead to some conclusions and recommenda- 
tions. 

Introducing Nine Projects 

There are two kinds of pertinent con- 
cerns which have attracted FIPSE support. 
Each is characterized by extraordinary com- 
plexity. One is the set of problems posed by 
technological innovation for state authorizing 
agencies and for accrediting agencies. The 
other is work toward the assessment of out- 
comes. 6f_tiVe_nirie projects I will mention^ 
eight were FIPSE-supported. 

Technolgical innovation, which brought 
us not only new educational programs but al- 
so realizable visions of electronic universities 
arid new institutions such as the recently 
founded National Tech nolgical University, 
forced us to face problems associated with 
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crossing jurisdictional lines. Basically, we fac- 
ed one big problem in three related parts, 
namely state authorization of programs, ac- 
crediting criteria, and the underlying defini- 
tion of legal presence in any state. Because of 
the linkages, an outcomes focus for accredita- 
tion would be only a necessary but hot a 
sufficient condition of success in solving the 
problem of delivering technologically innova- 
tion educational programs. 

Innovative approaches to learning faced 
a web or system of structural barriers, namely 
dealing with each state's and accrediting re- 
gion's requirements, which vary and involve 
time-consuming and costly processes and 
could be mechanisms for stifling outside 
competition, and a tangle of legal concerns 
involving constitutional issues relating to 
states' rights and interstate commerce and— 
most fundamentally— the absence of either 
any clear definition of "physical presence" to 
serve as the basis for state sovereignty or any 
body of pertinent case law. If we could hot 
surmount these problems, we would be limit- 
ed in our ability to reach learners and to reap 
the benefits of bur inventiveness. 

Segments of the £em_ began to re- 
spond. In 1973 the Ei jr^ Commission of 
the States published .100^ Jegislation for 
state authorization, and in 1978 tir Council 
of Postsecondary Accreditation (COir A) re- 
commended that the accrediting model be 
reconstructed to include assessment of edu- 
cational outcomes. It was COPA's perception 
that criteria focusing on the assessing of out- 
comes would be a key factor in promoting 
innovation. 

The next step, with FIPSE's support, 
was Project ALLTEL, "Assessing Long Dis- 
tance Learning Via JTelecommrnications,'' a 
joint project of the Council on P ostsecondary 
Accreditation arid the StateHigher Education 
Executive Officers (COPA-SHEEO). The pur- 
pose was to develop an encompassing strategy 
to ensure educational quality, promote the 
best use of the emerging technologies, and cut 
back the multiple accrediting and state 
authorizing activities. 



The summary report, which actually is 
long, detailed, and includes task force re- 
ports, was issued in October L j985. [t calls for 
^mmon procedures for authorization and ac- 
itation, thereby circumventing legal Is- 
sues. Secondly, it calls for a single and 
generally accepted format for institutional 
provision of information which indeed now is 
being field-tested. Thirdly, jt^ calls ; for the 
development of reciprocity among the states 
and the accreditors. The counfiy now has 
these principles and procedures and the pro- 
posed strategy for its consideration. The 
Implementation Task Fcrc* is promoting 
working agreements bciwrsn and among 
authorizing and accrediting bodies. In this re- 
gion, Florida^ Texas, and Virginia have de- 
cided to use the Institutional Profile. 

The fact that fi^ report bears Se Im- 
primatur of both COPA and SH E EG In itself 
makes it a landmark. Dissemination has been 
occurring, so that familiarization wkh the 
problems and proposals has been increasing, 
and the two organizations are continuing to 
work together, Their Accreditation Task 
Force has urged the adoption of outcomes 
measures and explicitiy makes the linkage to 
innovation. COPA-SHEEO's work adds to the 
overall thrust of the new Southern Associa- 
tion "Criteria" and provides a w^der set of sti- 
muli to innovation. 

All the other projects to which i_l will be 
referring address problems of organizing to 
measure outcomes. One project involves the 
Board cf Governors of the ^California Com- 
munity Colleges and the Accrediting commis- 
sion for Community and junior Colleges of 
the Western Association of Schools and 
Colleges. 

These two organizations have reached 
an agreement on their roles, L have conducted 
dozens of workshops to Improve Institutional 
and agency evaluation and planning activities, 
and have been addressing issues involving the 
measurement of learner outcomes. Their staff 
papers state that it is only through the evalua- 
tion of learner outcomes that colleges can 
monitor changes in the quality of Seir work 
and 4 eterm ' ne ff they are being effective. 
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They foresee testing for competencies, wish 
to contribute to the improvement of learner 
outcome measurements, and have developed 
an "Item Bank and Handbook 0 which is now 
being. tested by colleges as p&t of their ac- 
crediting self studies. 

In asking their selves what kinds of 
outputs/outcomes to seek, they considered 
institutional assessments weak along two di- 
mensions. They wanted to know much more 
about how many transferred and how they 
performed at the senior institution, how 
many were placed in jobs, and to what extent 
college work Helped people to become useful 
citizens. They found they also wanted to 
know what growth in skills, knowledge, and 
understanding of values students experienced 
while enrolled, and therefore wanted to note 
their characteristic: entiy , their work at the 
college, and the purpose to which they put 
that work after finishing their program. 
Furthermore, they contemplated the virtual 
certainty that the assessment of competencies 
would need to include a -vide Variety of tech- 
niques, among them interviews, performance 
examinations, review of experience, and 
standardized tests* and they stated that 
assessment would need to include the diagnos- 
ing of educational heeds. 

They were interested in feedback* be- 
lieving the results of outcomes-oriented evalu- 
ation should lead to decisions which improve 
the learning process. In order to enable col- 
leges to make comparisons with one another* 
they have proposed to make the Chancellor's 
office a depository for aggregated data, find- 
ing this approach both analytically effective 
and cost-effective. 

The enormous challenge would be to 
figure but appropriate ways to measure 
achievement* and especially to measure 
change. California community colleges may 
be the nation's most Varied set of institutions 
in student age, ethnicity and lifestyles* and 
maybe also in employment and residential 
patterns. There is a phenomenal range in stu- 
dent purposes* preparation and capabilities. 
Moreover* nearly all future enrollment in- 
crease will be of part-time students, and per- 



K ps their objectives differ from those of full- 
time students* e.g. perhaps most will be en- 
rolled to gain occupational skills. Not only 
are there mammoth problems in trying to as- 
sess change, but also it would be likely to be 
necessary to re-orient advk^nent in major 
ways. 



Indeed, the implications were Virtually 
sprawling in all directions. They perceived 
that a focus on outcomes would necessitate 
contemplating an amazing array of changes 
and innovations which would add up to 
fundamental institutional renewal, including 
changing the sense of responsibilities, chang- 
ing the mode of instruction, changing the 
capabilities of the teaching staff as wei! as the 
advising staff *. possibly hew articulation with 
nontraditional providers, renewal of pro- 
grams, improved management and coordina- 
tion, financial incentives for professional and 
organizational development* and ihter-institu- 
tional collaboration to prepare faculty and 
administrators. 

It may be fruitful for those interested 
in the Southern Association's hew "Cfiteria M 
to follow these developments in California* 
which incidentally extend to Hawaii, because 
aspects are n fre early stager of becoming 
operational. It may a!so be instructive to 
watch parallel developments in certain spe- 
cialized fields. 

I would like to mention two FIPSE 
projects rejated to the field of business. They 
illustrate different points. 

For perhaps, a dozen years the Ameri- 
can Academy of Collegiate Schools of Busi- 
ness (AACSB) has been contemplating 
outcomes measures, and some five years ago 
received a FIPSE grant for what was then 
called "The Accreditation Research Project" 
but is how known as "The Outcomes Mea- 
surement Project. 11 The original question, as I 
recall* was: What are the most reputable out- 
comes for the BBA and MBA? 

Now ready are two sets of instruments. 
One deals with seven sets of knowledge areas 
in the field of business. The other instrument 



86 



Groennings 



87 



focuses on skills, such as ability to communi- 
cate. 



AACSB w at the poinf of gathering 
data to determine national norms for out- 
comes. It is far from ready to put outcomes 
measures in accreditation standards, it is con- 
templating using outcomes measures as an al- 
ternative path to accreditation , so that there 
would be two acceptable processes, at least as 
an interim phase. 



AACSB thus has, as a matter of its own 
choice in trying to be constructive, a "hot 
potato": 

—the danger of being threatening to 
some of its constituency 

—the danger of teaching to the instru- 
ments 

—and interna! pressure stemming from 
the fact that AACSB includes in its 
membership both accredited and non- 
accredited schools. The latter are 
"chomping at the bit" for the opportu- 
nity to prove themselves in alternative 
ways, including Northeast Missouri 
State, which is a leader in advancing the 
value-added concept Clearly, the in- 
troduction of outcomes criteria would 
be a boon to non-traditional providers, 
e.g. the Arthur D. Little Management 
Education institute, which educates 
numerous foreigners under A.I.D. con- 

IfHCU. 

The second FlPSE project in the field 
of business illustrates how AACSB has been 
an engine of reform by setting accreditation 
standards. 

In 1974 AACSB stated that "The pur- 
pose of the curriculum shall be . . . preparing 
the student for . . . roles in business and 
society— domestic and worldwide." In 1980 it 
tightened the meaning: "There is no inten- 
tion that any single approach is required to 
satisfy the worldwide dimension of the curri- 
culum standard, but every student should be 
exposed to the international dimension 
through one or more elements of the curri- 
culum." 



As deans and faculties, wondered how 
to meet this hew standard, AACSB began a 
series of seminars to help faculty teach inter- 
national courses, namely, international fi- 
nance, international marketing, international 
management, international accounting, and 
introduction to international business. 

The AACSB international standard calls 
for reaching every undergraduate business 
student, but, like the Southern Association of 
Colleges and Schools, AACSB doesn't tell 
anybody how to do it. It is at this point that 
the new rules of the game stimulate innova- 
tion. 

In the case of the AACSB standard, the 
strategic choices are to incorporate inter- 
national business concepts, into the core of 
functional courses or to offer separate inter- 
national business courses. An institution 
might offer an international core course, a 
global perspective throughout the core 
courses, international modules within the core 
courses, or required international courses 
within the student's specializations. Any of 
these approaches would meet the AACSB 
standard. - 

It became apparent that there would be 
a need for models, dissemination, and finan- 
cial support for innovation in internationaliz- 
ing the business curriculum. 

Accordingly, it was appropriate for in- 
stitutions to turn to FlPSE, arid thus FlPSE 
could play a supportive role for the imple- 
mentation of the AACSB standard. 

FlPSE decided to support a three-year 
project involving a consortium of small uni- 
versities in the Northwest, with Pacific 
L jtheran University at the center, to develop 
jo.ntly a comprehensive program to inter- 
nationalize their curricula, including inter- 
disciplinary cooperation arid the development 
of nine modules for inclusion in all the non- 
quantitative undergraduate core business 
courses. FlPSE was riot supporting any major 
university's traditional strategy involving the 
hiring of experts to prepare specialized 
courses arid to produce experts for specialized 
institutions such as international banks. It 



87 



88 



Groennfngs 



sought to support a significant model involv- 
ing small institutions which do hot have a 
doctoral : ' ogram, and which would prefer to 
pursue a modular infusion strategy. These 
small institutions would not be hiring hew 
faculty, and it is not their function to pro* 
duce experts. Their students would be hired 
by small and middle-sized firms and would be 
expected to become familiar with doing in- 
ternational business as part of doing American 
business more broadly. The desired curricu- 
lum would also be useful in developing work- 
shops for employees of local firms. 

The challenge inherent in this strategy 
is two-fold. The first challenge is developing 
the modules, and incidentally, the appeal of 
this odular infusion strategy has become 
apparent as more than 40 institutions have re- 
quested the modules. The second challenge is 
retooling the faculties, i.e. one could not 
change the teaching and learning to meet the 
standard without faculty development 

Also along the international dimension, 
hew standards are rurig developed iin the 
foreign language field as the globalization of 
the economy forces greater attention to prac- - 
ticaJ communication and therefore jp foreign 
language oral proficiency testing. The prime 
Tiotor in this case has not been the accredit- 
ing agency, but a professional association. 
With the Exxon Education Foundation, 
FIPSE is supporting the development of 
foreign language proficiency testing. What if 
important for our focus today is that this case 
also shows the inevitably broad catalytic ef- 
fect of a change in expected outcomes. This 
will illustrate that such a change necessarily 
will induce innovations. 

The American Council on the Teaching 
of Foreign Languages (ACTFL) states that 
"Measurable foreign language proficiency is 
integral to the advancement of the profession 
and essential to its credibility. 11 

Testing instruments are being develop- 
ed and tested for the most common languages 
and will be developed for more. We are in the 
early stage of establishing z r ational system of 
testing for skil's ^tZaiiirr^i:^ &\vcn the long 



experience of the Foreign Servic institute 
with proficiency testing and the c ful work 
done in developing approbate me ics, I be- 
lieve this will be impressively succe ful. I am 
confident also because the assessn rit of a- 
chievement involves meeting an absolute 
standard at each of the five levels of mea- 
surement; ACTFL's program does hot 
attempt to measure individual progress in the 
sense of value added from any starting point, 
but only absolute student achievement, i.e. 
levels of competence. 

This will be a fundamental change with 
profound implications, jt will signal the im- 
portance of learning a foreign language as a 
skill to be used. If will motivate and enable 
students and teachers to earn credentials 
based on recogmred standards, while provid- 
ing yardsticks. *f progress. Moving the 
measurement of achievement from a semest- 
er-passed criterion to one based on profi- 
le y will allow people to proceed toward 
credential ing however and wherever they 
choose, while fostering continuity and link- 
ages across levels of ^education. There will be 
the addftSonal benefit of establishing bench- 
mifirs *F xcountability for teachers. 

fto may anticipate that the establish- 
ment of such a system will have catalytic 
effects on curricula: development and evalua- 
tion, on the design of teaching materials, and 
on parallel testing in increasing numbers of 
languages. The point is the same profound 
point being made de facto by the Southern 
Association: changing the scorekeeping chan- 
ges the game. 

In the case of ACTFL, we are talking 
about competency or outcomes testing, and 
immediately we started talking about all the 
hew issues: How could the tests be developed 
and administered? How would standards be 
maintained as universal constants when the 
number of testers i». ureases across the coun- 
try? Which languages and substantive fields 
would be included? What emphasis should be 
placed on diagnostics, which might help 
leathers overcome weaknesses? Is the vision 
sufficiently encompassing? Where would it 
be best to do the pilot testing and initial 
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evaluation? What should be the characteristics 
of the organizational mechanisms? Should we 
anticipate regional testing centers to train 
testers and administer tests? How should dis- 
semination be handled? How do we re-tool 
faculties? If different agencies are to be in- 
volved for different purposes, how should 
their efforts be coordinated? How would the 
system be financed? What sort of legitimiza- 
tion and organizational politics are needed to 
ensure a successful launch? 

All these questions arose as the foreign 
language field decided it should pursue a 
different kind of learning outcome and insti- 
tutionalize proficiency measures, and some 
such questions become new points of depart 
ture for innovation. 

If the objective is to measure an institu- 
tion's impact on students very broadly as 
whole persons, the measures will have to have 
multiple dimensions. There are three promi- 
nent models, provided by notably dissimilar 
institutions: Alverno College, Northeast Mis- 
souri State University (NMSU), and the Uni- 
versity of Tennessee. Long before my tenure, 
FiPSE supported early work with the State of 
Tennessee and was closely associated with the 
developments at Alverno. NMSU, which did 
not havie the hen*r outside funding, won 
the Ar teriian As>v don of State Colleges 
and U* Jes' 1983 Mitau Award for inno- 
vation ut:d change in state colleges and univer- 
sities. 

Alverno overhauled the whole curricu- 
lum, focusing it on eight skills outcomes: 
effective communications, analysis, problem- 
solving, Values in decision-making, social 
interaction, responsibility for the environ- 
ment, involvement in the contemporary 
world, and aesthetic response. The objective 
was to foster "abilities that last a lifetime." 
Each of the basic abilities is defined in six in- 
creasingly complex levels and these are in- 
fused throughout a curriculum which is 
otherwise organized in a traditional manner 
substantively. Facu jty teach toward the 
different levels of abilities in their substantive 
courses; in effect, tfieir responsibility for 
general student development parallels their 
responsibility in their field of knowledge. 



Alverno uses a variety of assessment 
techniques and has established both ah As- 
sessment Center, which administers assess- 
ments, and a faculty Assessment Council to 
provide technical assistance to faculty. There 
is also ah Office of Research and Evaluation 
which conducts research into student de- 
velopment and processes of teaching and 
learning and which evaluates the curriculum. 
At AlVerhbi everybody is involved, and a 
thorough evaluation of the whole program has 
confirmed that it is a stunning success. It has 
involved exceptional leadership and con iit- 
ment, and is hot readily replicable, although 
work in the same direction is how underway 
with FIPSE support at Clayton Junior College 
here in Georgia. I suppose that the Alverno 
approach could only evolve at a small liberal 
arts college. Alverno happens to be a Catholic 
women's college in Milwaukee with an en- 
rollment of 1,500. I suppose further that if 
institutions are going to test for broad out- 
comes, the testing system itself will be a boon 
to libera! education. 



In contrast to Alyerrip^jfte University 
of Tennessee at Knoxville (UTK) is a multi- 
versity with sixteen colleges and schools, a re- 
search institution w»th strong departments 
and corresponding decentralization f author- 
ity. The prod to its activity has been internal 
financial incentives provided by "performance 
funding" in statewide budgeting. There was 
an earlier external stimulus to developing 
"performance funding" when FIPSE original- 
ly co-funded, with the Kellogg and Ford 
Foundations, the Tennessee Higher Education 
Commission's pilot project designed to add a 
performance feature to the appropriations 
process. Currently, five percent of the overall 
budget is distributed to institutions on the 
basis of success in meeting performance crite- 
ria. UTK gained Kellogg funding to develop 
assessments of learning outcomes consistent 
with the criteria. It has developed a Value- 
added component for general education as 
well as projects in a substantia! number of de- 
partments to experiment wth measuring out- 
comes in the disciplines, plus instruments for 
assessing student and graduate satisfaction. A 
Lc "ng Research Center provides assistance 
a the campus. 
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Northeast M issourj State University, 
structurally a somewhat typical regional state 
university, L u se s sUndardized testing instru- 
ments in assessing value added for its entire 
stu dent b ody of 7,000. There are se vera I 
phases to ^is activity, both general [and field- 
specific, and there is a quest for patterns that 
make a difference in jearnihg. Costs are kept 
down by using existing instruments, and the 
University cites its efforts to focus ^ori i learning 
and oh tfte development of the individual 
student in making its general budget requests. 
The success of this program also is attrar ng 
na*k<al attention. 

Thus, there is a mosiac of activity a- 
cross the country. These activities, when 
viewed as a whole, seem a bit unsettling and 
chaotic, but tine chaos .ias momentum. 

Conclusions 

For this conference I was asked to draw 
on my experience with FjPSE to address the 
implications of new "Criteria" fcr technoSc- 
gical innovation and new approaches. 

It seems to me Sat the implications are 
far-reaching The focus on outcomes broadens 
the basic concept of excellence. L The role of 
accreditation will become increasingly ^impor- 
tant in the quest for quality and as a catalyst 
to innovation, and there will be a new dyna- 
mic tension between accreditors and institu- 
tions. Stich tension could be positive. 

I am impressed not only by the range 
of methodological problems in constructing 
instruments but also by the prospects of 
systematic complexity within and beyond 
academic institutions, considerable cost, and a 
heed to encourage initiative in institutional 
response to the "Criteria." 

We will enter an era of groping for as- 
sessment tools in both the institutional and 
specialized accrediting agencies. It will be re- 
latively easy to draw fac i le i nferences from 
the placement of students beyond the cam- 

P as i !* ^ appropriate 'Q 50 ! 11 ? 03 ^ 5 ^° 
pu rsue absolute standards in specialized fields. 
It will be especially difficult to cope with the 



? once _P t _ °f Y a jue added, which involves itera- 
tive measurement, and with trying to measure 
general outcomes. Some approaches may 
arouse general leeriness. 

It may be tempting for some beyond 
the academy to think that an outcome focus 
will help the colleges and universities to do 
more with less, since the objective is not the 
acquisition of resources but is instead their 
effective use. It seems clear, to the contrary, 
that the sh (ft to an ou tc om es f ocu s cou Id be- 
come very expensive, as it suggests more than 
the construction of measuring instruments. It 
can involve faculty training and more exten- 
sive counseling as well as management costs, 
all along new dimensions. Therefore, the im- 
plications fn innovative approaches can be 
virtually ubiquitous. 

We open the door to complexities when 
we make a seemingfy straightforward and 
reasonable decision to base institutional ac- 
countability at least in part on demonstrating 
progress in advancing student learning. It 
'instructive, I submit, to suggest to 
jots that such a change, which 
-Buaiiy addresses institutional mission, 
n to be broad in its impact and impli- 
es., - is. At a time of emphasis on both educa- 
tional quality and fiscal constraint, legislators 
n^ay want to say that the purpose of addi- 
tional appropriations will N to improve 
quality^ As some legislators may wonder 
about the effectiveness of the expenditures, 
we could face two legislative problems. One 
might be legislator concern about how satis- 
factorily to assess whether improvement has 
been [ occurring. The other is the related 
specter of an unfolding, changing relationship 
of academic institutions to state governments 
which hitherto have stayed a m med- 

dling in the assessment of academic perfor- 
mance. The latter Is another "quicksand" or 
"sleeper" problem or accreditors as well as 
campuses. Wisely, it has been the American 
tradition to leave issues of assessment funda- 
mentally in the hands of institutions and ac- 
crediting agencies. There may be a need for a 
broad conversation, in this new context, on 
institutional relationships as well as on as- 
sessment 
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Recommendations 

Each panelist a? Us- stsif 5 .- .vas 
asked to consider offering a reco?* HV'< Mtion 
or two. 

Recommendations can be fr *iriiy of 
two kinds: The first concerns ins hi . nsand 
programs. They will heed to clarify their mis- 
sions and develop ways to assess learning out- 
comes and institutionalize the outcomes fo- 
cus, The second concerns the accreditors, 
which will heed to contemplate being helpful 
to institutions in hew ways as well as how to 
evaluate the achievement of institutions. 

My assigned topic, however, is the im- 
plications for innovation. I have tried to make 



the point that the innovations propelled by 
the Southern Association's hew "Criteria" ctri 
be far-reaching, varied from orie institution to 
ihoihf in accordance with their varying mis- 
sions, arid costly, implementation will involve 
experimentation and time, institutions will 
seek models, information and finar?~ial help. 
In time, credibility will be "on the line", and 
in the meantime there could be del^y> attri- 
butable to concern that other resources will 
be traded away in order to meet the expense 
of launching this hew undertaking. The South 
is embarking upon a Venture. Yet unlike 
corporations, colleges and universities do not 
have venture capital. I recommend that there 
be consideration v* creating incentive 
funders. 
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"ASSESSMENT^ EVALUATION, AND ACCREDITATION" 
Third Annual Conference on Research in Postsecondary Education 



GEORGIA CENTER FOR CONTINUING EDUCATION 
UNIVERSITY OF GEORGIA 
ATHENS, GA 396G2 



JANUARY 15-16, 1986 



Sponsored by: 

THE UNIVERSITY OF GEORGIA 
institute of Hi^?r Education 
Office of Institutional k. search and Planning 
Center for Continuing Education 



HIGHER EDUCATION IN GEORGIA: 
ASSESSMENT, EVALUATION, AND ACCREDITATION 



Wednesday, January 15th 
12:00 - 1 :00 p.m. Registration 
1 :00 - 2:15 p.m. Opening Session 



"The Emerging Role of Asses-v^c^t and Evaluation in 
Postserondary Education 1 ' 

Cameron Fincher 

institute of Higher Educetion 

2:15 - 3:30 p l/>Qurta Booth, Chair 

irst Panel on Assessment 

"Assessing Student Achievement" 

joe Marks 

Southern Regional Education Board 
11 Assessing Educational Outcomes" 
Tom Red m on 

Southern Regional Education Board 
"Assessing Vocational Competencies" 
Michael McCord 

Athens Vocational /Technical School 

3:30-3:45 p.m. Break 

3:45 - 5:00 p.m. Claire Swann, Chair 

Second Panel on Assessment 

"Needs Assessment in Programs for Health Services Personnel" 
Lib by Morrte 

Institute of Higher Education 

"Assessing the New Standards for High School Graduation and 
College Admissions" 

Nathaniel Pugh 
Georgia Southern 

"Needs Assessment Sh Vocational Programs" 

Ken Allen 

Georgia State University 



5:00 - 6:30 p.m. Conference Recess 



ERIC 



6:30 p.m. Dinner Session 

••The New Criteria for Accreditation" 

James T. Rogers 

Commission on Colleges 

Southern Association ot 4 Schools 

Thursday, January 16th 

8:30 - 10:00 a.m. Katherine Boardnian, Chair 

Panel on Implications of SACS 9 New Criteria for: 

•'Institutional Research" 

Larry Jones 

Office of institutional Research & Planning 
••Developmental Studies" 

Division of Developmental Studies/UGA 

"Instructional development" 

Ronald Simpson 

Office of instructional Development/UGA 

"Admissions" 

John Albright 

Office of Admissions/UGA 

10:00- 10:15 a.m. Break 

10:15 -11:30 a.m. Rick Rose, Chair 

Panel on Implications of SACS' New Criteria for: 

"Learners" 

Margaret Holt 

Department of Adult/Continuing Education 

"Higher Education's Changing Clientele" 

Herman Smith 

Visiting University Professor 

Institute of Higher Education 

"Technological innovation and New Approaches" 

£veit Groehnings 

^!ilnS_PT^ fessor 

institute of Higher Education 

1 1 :30 - 12:15 p.m. Open discussion and recommendations for assessment and 
evaluation in institutional accreditation 
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